Use the debugger first. Debug + Break All, Debug + Windows + Threads to see what threads are still running. You can double-click one and use Debug + Windows + Call Stack to see what it is doing. The typical case is a thread you started but forgot to tell to terminate. The Thread.IsBackground property is a way to let the CLR abort a thread automatically for you.
Technically it is possible to have a problem with a device that prevents a process from shutting down. The Threads window would then typically show only one thread with an otherwise impenetrable stack trace. If you use Task Manager, Processes tab, View + Select Columns, tick Handles, then you may see only one handle still in use. The diagnostic then is that you have a lousy device driver on your machine that doesn't properly support I/O cancellation. Which could leave a kernel thread running that doesn't quit, preventing the process from terminating. Very unusual, look for the reasons given in the first paragraph first.