Home / DE Associate practice test / Data Ingestion and Loading / Question

Databricks Data Engineer Associate · free practice question

An Auto Loader stream ingests from a bucket that accumulates millions of files. Directory-listing mode has become slow and expensive because each trigger…

Exam-style question on Data Ingestion and Loading from the free DE Associate practice test, with the answer and a full explanation below.

Updated November 10, 2026

Data Ingestion and Loading

An Auto Loader stream ingests from a bucket that accumulates millions of files. Directory-listing mode has become slow and expensive because each trigger re-lists the whole path. Which change most directly reduces the per-trigger discovery cost?

  1. ASwitch to file notification mode, where Auto Loader subscribes to cloud event notifications and learns about each new file from the queue instead of repeatedly listing the entire large directory on every trigger.
  2. BIncrease the trigger interval so that the directory listing runs far less frequently, which spreads the fixed listing cost out over time and therefore effectively eliminates the per-trigger discovery expense.
  3. CDelete the checkpoint directory before each run so that Auto Loader forgets all of the files it has already tracked and consequently has a far smaller set of paths that it then needs to list on each following trigger.
  4. DSet cloudFiles.maxFilesPerTrigger to a very large value so that every outstanding file is discovered and pulled in during one single directory listing rather than being spread across many smaller batches.
  5. EConvert the incoming source files to Delta format so that Auto Loader can read the transaction log directly and therefore no longer has to discover individual files on the cloud storage landing path.
Answer & explanation

Correct answer: A

WHY A: File notification mode subscribes to cloud storage event notifications, so Auto Loader learns about new files from a queue rather than re-listing a huge directory each trigger — the standard fix for high-file-count paths. WHY NOT B: A longer interval reduces how often listing runs, but each listing is still a full-directory scan and stays costly; it does not eliminate listing. WHY NOT C: Deleting the checkpoint causes already-processed files to be reprocessed and duplicated, not cheaper discovery. WHY NOT D: maxFilesPerTrigger only caps files per batch; it does not change the discovery mechanism. WHY NOT E: Converting to Delta changes the source format, but the cost here is listing files on the landing path.

Take the full DE Associate practice test free →

Would you pass today?

This question is one of 45 in the free, timed Databricks Data Engineer Associate practice test. Take it and get a section-by-section score.

Start the free test →