Since you've stated that your templates are supposed to be rendered server-side (and thus included as part of your server-side code):
You appear to be using Apache or some similar server process to process your code. Most such servers will only apply PHP inclusion (your first two examples) or SSI inclusion (your second two examples) to files with specific extensions. For example, if you've got a file whose name ends in ".ejs", most servers will try to directly serve it without running it through a PHP interpreter (and thus, any <?php...>
directives will be passed through literally, rather than processed as inclusions) or applying SSI directives to it. Standard rule for servers is to run a file through PHP if its name ends in ".php" (possibly also including ".php4" or ".php5") or to run it through SSI if its name ends in ".shtml" (or sometimes just ".html").
The solution is to (if you can) configure your server to parse ".ejs" files by PHP or SSI. If that's not possible (and it might not be under typical hosting arrangements), your best bet is to preprocess your source files at development/deployment time using some sort of Web-oriented preprocessor or a general macro processor (e.g M4, should be automatically installed on any Unix/Linux/MacOS system, can be easily installed on any Windows system).