[Marketo, Marketo, develops marketing automation software that provides inbound marketing, social marketing, CRM, and other related services. , Features, Featurexa0Namexa0xa0xa0xa0, Supported, Notes, Capture Deletes, check, LEAD, LIST_MEMBERSHIP , OPPORTUNITY_MEMBERSHIP and , PROGRAM_MEMBERSHIP, tables. We read the activities stream, and once we see a delete activity, we mark the record as deleted. , Custom Data, check, LEAD , COMPANY and all activity tables. For each activity, we create a table in the format , activity_{activity_name}, Data Blocking, check, “Column level, table level, and schema level. We dont allow blocking for the “, LEAD, table., Column Hashing, check, Re-sync, We have disabled re-sync from the Marketo connector UI because it can take months. Contact support if you wish to run a full re-sync., History, check, EMAIL_TEMPLATE_HISTORY, table only, API Configurable, check, Priority-first sync, check, All activity tables, dbt Package, check, Get the package: , source, / , model, Setup guide, Follow our step-by-step , Marketo setup guide, to connect Marketo with your destination using Fivetran connectors., Sync overview, “We use Marketos “, REST API including the Bulk Activity export. Initially, we sync the lead and activity tables and then sync the remaining tables alphabetically. Our sync strategy for activities data varies depending on which version of our Marketo connector you use. See the , Sync strategy for activities, section for more information., “Because accidentally activating a historical re-sync can cause lengthy disruptions, weve disabled the historical re-sync button in the Fivetran dashboard for this connector. Contact “, our support, to enable a full historical sync., Sync strategy for activities, Priority-first sync, “We use priority-first sync to pull your activities data. Priority-first syncs fetch your most recent data first so that its quickly ready for you to use. “, During your initial sync, Fivetran fetches your most recent seven days of data., “In all subsequent syncs, we continue to fetch your most recent data first. We also sync historical data until we reach the end of your activities data retention period. For example, if Change Score activities are retained for 90 days, we will sync 90 days worth of Change Score data.”, Priority-first syncs allow us to keep your most recent data up to date while syncing your historical data over time., BULK API, The latest version of our Marketo connector uses , only, BULK API to fetch activity data, because BULK API guarantees data integrity. This connector uses REST API to fetch all other data., If you are using our latest Marketo connector, make sure you purchase sufficient BULK API quota from Marketo., When Fivetran requests data from Marketo via BULK API, we can provide a start date and an end date for the range we want to sync using the , sinceDatetime, and , untilDateTime, query parameters. In this way, we fetch activities for a defined time range, save the state, and request the next time range. Because we can save state accurately, we preserve data integrity even if syncs fail or are interrupted., Combined BULK and REST API, Earlier versions of our Marketo connector use REST API to fetch all data but activity data. These connectors use a combination of BULK API and REST API to fetch activities. , BULK API is more reliable and is the preferred method whenever possible. The BULK API part of the combined strategy works exactly the same as the , BULK API-only strategy, of the latest version of our connector., When Fivetran uses REST API to request activity data from Marketo, we can request a start date using the , sinceDatetime, ” query parameter. However, unlike with BULK API, we dont have the option to specify an end date, so we have to rely on sorted records to save the state. That is, when we see an activity, we store its timestamp in the state. We do this to keep track of the time until we have fetched the activities. “, “However, Marketos activities endpoint does not always provide the records in sorted order. When Marketo activity records are not in sorted order, we cant save the state correctly. If the connection fails or we have to reschedule because we reach the API quota limit, we might end up saving the wrong state and missing some records.”, “Why dont we simply wait to save the state until we have fetched all the activities instead of attempting to store the intermediate state? If we dont store the intermediate state and the sync fails or is rescheduled, we have to start the entire sync from the beginning. Because Marketo has large amounts of data and its API is already very slow, that could take an extremely long time. Moreover, we could reach the API quota limit repeatedly without finishing the sync, and get stuck in a loop. Saving the intermediate state is a necessary, if imperfect, workaround to syncing through the Marketo REST API. “, Leads replication and Marketo Corona, Fivetran uses the , updatedAt, query parameter to retrieve only the new and updated data from the Leads API endpoint. However, this feature is only supported for those accounts that have enabled Corona. , To allow us to perform incremental replication of your leads data, contact Marketo to enable the feature for your account., Sync strategy for PROGRAM_MEMBERSHIP table, When we sync the activity tables we obtain the program IDs from the activities. We then use the program IDs to fetch details about the programs. We individually check all the programs and then sync all the leads that belong to a specific program into the , PROGRAM_MEMBERSHIP, table., Sync notes, “We use Marketos “, Get Deleted Leads, API endpoint to fetch the deleted leads. Marketo retains the activity details of deleted leads for only 14 days. See , “Marketos REST API documentation”, for more information., If you are using the latest version of our connector (v3), we fetch the , deleted_lead, activity before all the activities. If you pause your connector for more than 14 days and Marketo removes the , deleted_lead, activity, we may not be able to identify some of the leads as deleted., If you are using the previous version of our connector (v2), we fetch the , deleted_lead, ” activity along with all the activities. If your connector doesnt complete a successful sync in 14 days, we may not be able to identify some of the leads as deleted because Marketo pruned the activity details.”, Schema information, “Marketos schema follows Fivetrans “, standard API schema rules, ., “Extracting data from Marketo can leave you with sparse, duplicative, and confusing tables if youre not careful. Fivetran helps to organize this data into a relational data model, making your queries more efficient, your destination less cluttered, and your data easier to analyze.”, Marketo supports creating custom tables. The custom tables that can be in your Marketo account can vary depending on which integrations you have, and how you have configured it. Most of the custom tables are basically custom event tables. For example, “Event X” becomes “activity_event_x” in our system. We will make tables for the activities youre trying to track. , Schema, This schema applies to the latest version of our Marketo connector, released in private preview in November 2019, and applies to all connectors created today , Beta, To zoom, open ERD in new window, This schema applies to the generally available version of our Marketo connector, and all versions of our Marketo connector created before November 1, 2019., To zoom, open ERD in new window]