In the late 19th century, Karl Weierstrass invented a fractal-like function that was decried as nothing less than a “deplorable evil.” In time, it would transform the foundations of mathematics.

Math is, by definition, a subject about numbers. But at the National Council of Teachers of Mathematics this week, math educators said the subject has its own language, too—and knowing how to speak it ...