I have recently deployed a data model on Astera DWBuilder, everything was completed and verified. It deployed successfully. I have created some data pipelines that fetches data from all the sources and feed into my data warehouse. These pipelines are scheduled to run on different frequencies.
I have noticed some pipelines which are inserting data into fact tables are failing, and the problem seems to be delayed loading of dimension tables due to the size of the data or queued jobs which results in null values being inserted into fact table as foreign keys.
How can I make sure that whenever the fact loader runs, it verifies the data in dimension loader first?