You may embed latex using a variety of different syntaxes. The shortcode (http://codex.wordpress.org/Shortcode_API) syntax is preferred. So [latex]E=mc^2[/latex] will work out of the box. This also forces loading of mathjax.
Additionally, you can use native mathjax syntax — $$E=mc^2$$ or (E=mc^2). However, if this is the only syntax used, the plugin must be explicitly told to load mathjax for the current page. This can be achieved by adding a [mathjax] shortcode anywhere in the post. For posts with both [latex]x[/latex] and $$x$$ syntaxes this is unnecessary.
You can use wp-latex syntax, $latex E=mc^2$. Parameters can be specified as with wp-latex but will be ignored. This means that mathjax-latex should be a drop in replacement for wp-latex. Because this conflicts with wp-latex, this behavior is blocked when wp-latex is present, and must be explicitly enabled in the settings.
You can also specify [nomathjax] — this will block mathjax on the current page, regardless of other tags.