salesforce - How to do Salesforce SOQL query to only select objects which have record counts greater than zero?

Question

I am trying to do a daily copy (daily snapshot) of all the Salesforce objects using Azure Data Factory (ADF) "Copy" activity into an Azure Datalake by first using a Lookup query activity (SOQL as shown below) and then doing a ForEach activity for the list of objects returned from this query:

SELECT qualifiedapiname 
FROM EntityDefinition 
WHERE IsQueryable = true 
  AND IsRetrieveable = true 
  AND IsDeprecatedAndHidden = false
LIMIT 2000

However, this query also returns objects which have no (zero) records at all and I want to skip them. How can we ?

Is there any field/property metadata in Salesforce's EntityDefinition that can tell if the object has any records in it or if its blank, something like "IsBlank = true/false" ?

As per this link:- SOQL doesn't allow count() on EntityDefinition object, therefore it's not possible to find if a certain object (qualifiedApiName) has any records in it from the SQL query:https://developer.salesforce.com/docs/atlas.en-us.api_tooling.meta/api_tooling/reference_objects_soql_limits.htm#topic-title

Running this on ADFv2 with batch count of 50; Auto DIU, and degree of parallelism = 32. Currently, all these SFDC objects have less number of records; but eventually when it starts growing - would have to re-design this pipeline anyways to segregate small objects as daily full copy and large SFDC objects with a one-time load and then incremental loads using either watermark table and/or max last modified date/time method.

Thanks!

标签： salesforceazure-data-factory-2soqlazure-data-lake-gen2