The simplest technique without relying heavily on an external library is prevalence. Periodically checkpoint by using serialization to take a snapshot of your state, then maintain a journal by serializing enough information on every side-effectful operation against your data to repeat it later. If something blows up, reload the most recent checkpoint, then re-apply all journal records written after that point.
For something more sophisticated, try software transactional memory. It may be somewhat clumsy to implement in current languages, but is quite powerful and may give you some additional concurrency techniques as well.
For irreversible operations like accessing a Web service or sending an email, you'll need to use compensating transactions: make another Web service call to cancel or update the results of the previous one, or perhaps send another email advising the recipient that things didn't work as intended.