それらを保持したい場合は、それらが表す非終端記号にそれらを付けたいと思います。おそらく、セットメンバーから、それらが FIRST または FOLLOW である非終端要素へのマップなど、逆も必要になるでしょう。
Then you error recovery routine can use previous or more likely the "next" input token (that's the one which caused you report an error) to decide what you might insert into the input stream instead.
I don't actually store these. I use a GLR parser, whose parse tables are essentially LALR parse tables, and simply build a recursive algorithm to crawl over the tables to see what tokens might allow the parser to proceed. Indirectly, I'm taking advantage of FIRST and FOLLOW, since those were used to construct the parse tables.
If you are taking a compiler design course, I'm recommend focusing on the post-parsing issues. You can sink tons of time into trying to "patch up" the source in response to an error, and all you'll learn is that a) this is hard, and b) nobody will particularly like the choice you offer them. You can spend energy on syntax repair until you're blue in the face but I'd wait until somebody asked you to do it for a job. In the meantime, for a compiler class, I'd let my compiler simply say, "Syntax error on line N" and abort.
Crummy but good enough to let you get on with the more interesting part.