The overhead of creating a thread is about 100 micro-seconds. i.e. if you do less than 100 micro-seconds of work you will have more overhead than work done and you program can be slower that being single threaded.
The overhead of create a task to an existing Executor service is about 2 micro-seconds. i.e. if the task takes less than 2 micro-second you may have more overhead than real work done.
If you have CPU bound process, you need about the same number of threads as core to keep all the cores busy, while minimising overhead.
e.g. if you have 8 cores, I suggest you combine the work done so you have 8 threads with one task each total. You can have more tasks than this but you may find it take longer to process.
Of course you should shutdown your ExecutorService when you have finished with it. The reason you don't see this done in all examples is that it can be a good idea to create one ExecutorService which is used for the life of the application.