Classification problems are all around. Decision trees are popular classification tools, deemed to be leaders in terms of interpretability. Classic decision trees consist of a recursive, and greedy, procedure. The use of a greedy strategy yields low computational cost, but may lead to myopic decisions. The latest advances in Mathematical Optimization have resulted in a growing research on building optimal classification trees. Mixed-Integer Optimization models have been recently proposed to tackle this problem. Although the results of such optimal classification trees are encouraging, the use of integer decision variables leads to hard optimization problems. In this talk, we propose a continuous optimization formulation instead. The flexibility we can borrow from this framework will be discussed.