I don't think there is any hope of reuniting again. The reason mostly lies on your ex. He claimed that he shouldn't have rushed back into the relationship. Maybe the flame he felt for you was quickly diminished. He wished to go back to his former ex.
You did handle the breakup maturely. Now, you will have to move on with your life.
Even if you get together, the relationship might lead to another breakup. This is because your ex had regretted rushing back to the relationship.
It will also be difficult to get back with your ex before you broke up to be with your childhood sweetheart.