You should definitely store your data in some persistent storage medium (SQL, Azure Tables, XML file, etc). The issues with storing items in memory are:
You have to find a way to store them once for the application and not once per user. Else, you will have potentially several copies of a 2-5 MB dataset floating around your memory space.
Users can add records, are these for everyone to see or just them. How would you handle user specific data.
If your app pool recycles, server gets moved by the Azure engineers, etc, you have to repopulate that data.
As said above, caching can really help to alleviate any SQL Azure latency (which btw, is not that high, we use SQL Azure and web roles and have not had any issues).
Complex queries. Sure, you can use LINQ to process in memory lists, but SQL is literally built to perform relational queries in a fast, efficient, data-safe manner.
Thread safe operations on an in-memory collection could be troublesome.
Edit/Addendum
The key, we have found, to working with SQL Azure is to not issue tons of little tiny queries, but rather, get the data you need in as few queries as possible. This is something all web applications should do, but it becomes much more apparent when using SQL Azure rather than a locally hosted database. Lastly, as far as performance/caching/etc, don't prematurely optimize! Get your application working, then identify bottlenecks. More often than not, it will be a code solution to fix the bottleneck and not necessarily a hardware/infrastructure issue.