1# To Setup Application:
2
31.  It is assumed that the Go runtime has been installed with a properly set
4    $GOPATH
52.  Run `make bootstrap`
6
7# To Provision a Database
8
9**Use Google Cloud**. This is one assumption made based on the idea that Google
10Data Studio will visualize the data.
11
121.  Log in to [the Google Cloud Platform](https://pantheon.corp.google.com)
132.  Under the **Storage** section, navigate to **SQL**
143.  Click **Create Instance**, select **MySQL** and hit **Next**
154.  Choose **MySQL Second Generation** (the default)
165.  Set Instance ID and root password, then click **Create**
176.  Wait for the instance to initialize, then navigate to **databases** and then
18    click **create database**; choose the defaults and make note of the chosen
19    database name.
207.  Navigate to **Users** and **Create user account** for the purpose of
21    creating a non-root user to log into the database; Make note of the username
22    and password chosen.
23
24The application assumes that both a development and production environment
25exist, therefore the above steps will need to be completed a second time. Now
26the following environment variables will need to be set in your application
27environment:
28
29*   GCP_DB_INSTANCE_CONNECTION_NAME_DEV: The instance ID of the provisioned
30    database server; this is listed on the SQL homepage in Google Cloud
31    alongside the respective database instance
32*   GCP_DB_NAME_DEV: the database name created from step 6
33*   GCP_DB_USER_DEV: the username created from step 7
34*   GCP_DB_PASSWORD_DEV: the password created from step 7
35*   GCP_DB_PROXY_PORT_DEV: an arbitrary, unique port used for the local secure
36    MySQL proxy
37
38The same environment variables should be set for production. The names are the
39same but replace **"DEV"** with **"PROD"**
40
41One the environment variables are set, the database can be readied by running:
42
43`make db_upgrade`
44
45To run the same set of upgrades for production, run:
46
47`ROLE="prod" make db_upgrade`
48
49# Running the Application
50
51Executing the application will by default:
52
531.  Clone all of the repositories specified in config.json and write the
54    observed differentials to the database
552.  Run application-specified analytics on the persisted data and write the
56    output to designated denormalized tables; those tables are tightly coupled
57    to Datastudio data sources in the [AOSP Branch
58    Differentials](https://datastudio.google.com/c/u/0/org/UTgoe29uR0C3F1FBAYBSww/reporting/1lTzMXHBWiVvc0Dnb2DJvRQeTVCAIuvGF/page/9rqP)
593.  Generate CSV reports based on persisted data
60
61Applications steps can be optionally disabled for presumably one-off runs:
62`ROLE="prod" ./repodiff
63[--execute-diff=false][--denormalize-data=false][--generate-report=false]`
64
65`config.json` will specify input and output parameters for the application;
66update to reflect your own environment.
67
68To run:
69
70`make run`
71
72For production: `ROLE="prod" make run`
73
74# Tooling
75
76To connect to the MySQL server used by the application, run:
77
78`make db_shell`
79
80To create new, canonical migration scripts to update the database schema, run:
81
82`make sql_script`
83
84To undo the latest database migration, run:
85
86`make db_downgrade`
87
88To upgrade to the latest database version, run:
89
90`make db_upgrade`
91
92To run tests, run:
93
94`make test`
95
96# Consuming the Application Output
97
98TLDR: Use [Google Data Studio](https://datastudio.google.com); Your datasource
99will be tables from the provisioned database set up in the provisioning
100instructions. All intended consumable tables are prefixed with
101**denormalized_view_**
102
103The rationale behind the application is that setup in Data Studio should require
104little to no learning curve (for developers especially), and views are simple
105projections of the underlying model. The application, then, should run whatever
106necessary logic to produce desired analytics that can be written to a
107denormalized database table for straightforward consumption.
108