Build your Agency Data Pipeline by Coding is for Losers

Build your Agency Data Pipeline

Bring your digital agency into the 22nd century

Overview

Wrestle your agency's data analysis workflow completely to the ground, by building a BigQuery data pipeline to house + analyze all of your critical data. Down with chaos!

At CIFL, we operate the Agency Data Pipeline service for a number of digital agencies - ranging from SEO to eCommerce, and in size from 5 clients to 50.

This course completely opens the kimono on how we operate that service, and allows you to build + operate your own Agency Data Pipeline.

The course includes:

A detailed walkthrough of how we built our own internal data pipeline at CIFL, pulling data from 8 APIs (Google Analytics, FB Ads, and more) to answer the question: which is the highest performing marketing channel for the CIFL blog?
Roadmapping exercises and instructions for building your own data pipeline, specific to the challenges of your work.
A one-hour coaching session with David, commissioner of Coding is for Losers, to make sure your data pipeline tracking plan gets put into action.

Many thanks to Supermetrics and Stitch for providing demo accounts for use in building the course.

With love,
David
Commissioner, Coding is for Losers

What's included?

38 videos 5 files

Contents

Intro to the Course Pipeline Project

WTF is ADP?

Copy the Tracking Plan Template

Check out the Demo Pipeline Tracking Plan

Check out the Demo Pipeline SQL Models on Github

Check out the Demo Pipeline Data Studio dashboard

Start your Pipeline with a Question

Intro to question asking

2 mins

Demo questions for the CIFL pipeline

7 mins

Translating your Question into a Tracking Plan

Meet the Tracking Plan

5 mins

Mapping out your Data Sources

6 mins

Planning your Data Models

7 mins

Scoping out each Site or Client

5 mins

Firing up your Data Feeds

Intro to data feeds

2 mins

BigQuery initial setup

4 mins

Setting up your BigQuery tables

7 mins

Pushing data from Sheets to BigQuery

6 mins

Supermetrics quickstart for beginners

3 mins

Pulling data from unsupported APIs into the Tracking Plan

9 mins

Using Stitch for Automated Data Feeding

Stitch initial setup

3 mins

Pulling GA data using Stitch

7 mins

Pulling Adwords data using Stitch

4 mins

Pulling FB Ads data using Stitch

4 mins

Writing SQL queries to answer your questions

Intro to DBT for SQL data modeling

4 mins

Creating your DBT project

8 mins

Connect your BigQuery database to DBT

8 mins

Writing your 'processing' level SQL queries

9 mins

Writing your 'join' level SQL models

6 mins

Sidenote: on debugging DBT models

7 mins

Pro tip: standardizing URL structure

3 mins

Pro tip: using DBT macros

6 mins

Writing your 'admin' level SQL models

3 mins

Writing your 'math' and 'visualization' level SQL models

8 mins

Visualizing your analysis in Data Studio

Intro to visualization with Data Studio

3 mins

QCing your raw data in Data Studio

6 mins

Reaching the summit: answering your question

11 mins

Answering ancillary questions

4 mins

Pulling data from BigQuery into Sheets

4 mins

Putting your Pipeline into production

Intro to productionalizing your pipeline

2 mins

Pushing your SQL models to Github

3 mins

Using Sinter to run your SQL models on a schedule

5 mins

Scheduling your data pipeline

5 mins

Testing changes to your data pipeline

3 mins

QCing data using Supermetrics as a check

5 mins

Congrats!

Wapow! You made it.

2 mins

FAQs

Q:

Is the course only for digital agency owners?

Nope - if you work with sales + marketing data in any way (Google Analytics, FB Ads, SEMrush, a CRM, Shopify, etc) in your work, you can benefit from data pipelining.

If you don't analyze digital marketing + sales data in your day-to-day work, I'd recommend not taking this course.

Q:

How much do the tools cost to build my pipeline?

The tools required to build your pipeline are:

Stitch (Free up to 5mm rows pushed to a database, $100 / month and up after that) or Supermetrics ($50 / month) for pulling data from APIs
Google Sheets + Apps Script (Free)
Google BigQuery (or similar database) for data warehousing (free 12month trial + $300 credit, cost varies by usage after that)
DBT to model data using SQL (Free + open-source)
Sinter to run DBT models on a schedule (Free for up to 6 runs a day, $100 / month for unlimited runs)

So the bottom line is - the cost of your pipeline depends on the size of your pipeline. If you're pulling + storing a small amount of data, it may be completely free.

If you're pulling a large amount of data, it will be more expensive. The raw tool cost for most of our large Agency Data Pipeline implementations is around $500-$600 a month.

Making Spreadsheets Great Again

Helping you wrangle your data + automate your work, without (hardly ever) leaving Google Sheets.