Understanding Data Stream

  • Updated

What Is Data Stream?

Data Stream is Demandbase’s customer-data delivery service. With this service you can have your Demandbase customer data streamed daily to your data lake, warehouse, cloud service provider, or directly to your BI tools, such as Domo, Tableau, and PowerBI. Having access to your Demandbase customer data within your own environments gives you the flexibility to enable:

  • Dashboard development
  • Integrated data science research
  • Machine Learning model development
  • Business Intelligence integration
  • Aggregated analysis with other information

With Data Stream you no longer need to complete the daily export and import tasks. All of the data is automatically delivered to you. You can store all the Demandbase data in the same place as all other data in your company, giving you a holistic view that allows you to gain insights from the combined dataset. For more information, see the Data Stream Specification document.

Two Access Methods: Delivery and Hosted

Data Stream has two methods of access: Delivery and Hosted. Choose the best option for your organization’s needs.

Delivery

Daily Data Delivery to Your Cloud

Data Stream delivers Demandbase data to your cloud account. Data encompasses your account’s Platform data plus more, such as account scores, in an easy-to-ingest entity-based structure.

We deliver the latest data daily by 12pm UTC (7am EST) with up-to-date current values and historicals. Once you receive the data, you fully control its data policies such as retention and permissions.

Cloud-Native Storage & Data Warehouse Destinations

Data Stream is all about delivering to cloud-native data warehousing and storage destinations. These solutions offer unparalleled access, scale, and performance. We continue to add support for additional cloud-native destinations, so if you don’t see yours supported, reach out to your account team.

We store data as delivery-day partitioned tables in all warehouse destinations. By default, we store data in the Apache Parquet format in all storage destinations, separated by delivery-day directories. CSV file format is an available option upon request. Our supported destinations are:

  • Data Warehouse Destinations: Google Cloud Platform (GCP) BigQuery, Amazon RedShift
  • Data Storage Destinations: GCP Google Cloud Storage (GCS), Amazon S3, Microsoft Azure Blob, Demandbase Hosted SFTP

Delivery Setup

Demandbase will help guide your infrastructure or engineering contact(s) to set-up delivery access to the chosen data destination in your cloud account. 

The destination requires only a reserved location (for example, BigQuery Dataset, S3 bucket) for daily deliveries. In addition to setup assistance, we provide detailed instructions for all steps. Setup and initial delivery can be completed in a day for currently supported destinations. Note that delivery to only one cloud account is supported per customer.

Hosted

Daily Data Refresh in the Cloud

Data Stream hosts data in the Google Cloud Platform or in a Demandbase Hosted SFTP server for your access and retrieval. Data encompasses your account’s Platform data, such as account scores, in an easy-to-ingest entity-based structure.

We refresh the data daily by 12pm UTC (7am EST) with up-to-date current values and historicals.

Storage File (GCS API) or Batch Query (BigQuery API) Retrieval 

For connecting to Data Stream with BI and Reporting Tools, we recommend daily batch retrieval via the GCP BigQuery API. BI Tools such as Domo, Tableau, and PowerBI support scheduled daily retrieval of Data Stream via BigQuery connections.

For retrieval of raw data files, we recommend retrieval of daily refreshed files via the GCS API. We store files in the Apache Parquet format.

Each method of hosted access offers fully customer-federated data and access via separate BigQuery Datasets and GCS buckets, respectively.

GCP Access Account Setup

Data Stream supports both Google user account and GCP service account access for one dedicated account per integration. These accounts are used by the customer technical contact(s) responsible for integrating with Data Stream. You can switch or suspend access accounts upon request.

SFTP Access Account Setup

A technical contact name and email for correspondence is required. Access is done via the SSH protocol.

Was this article helpful?

0 out of 0 found this helpful