1# To Setup Application: 2 31. It is assumed that the Go runtime has been installed with a properly set 4 $GOPATH 52. Run `make bootstrap` 6 7# To Provision a Database 8 9**Use Google Cloud**. This is one assumption made based on the idea that Google 10Data Studio will visualize the data. 11 121. Log in to [the Google Cloud Platform](https://pantheon.corp.google.com) 132. Under the **Storage** section, navigate to **SQL** 143. Click **Create Instance**, select **MySQL** and hit **Next** 154. Choose **MySQL Second Generation** (the default) 165. Set Instance ID and root password, then click **Create** 176. Wait for the instance to initialize, then navigate to **databases** and then 18 click **create database**; choose the defaults and make note of the chosen 19 database name. 207. Navigate to **Users** and **Create user account** for the purpose of 21 creating a non-root user to log into the database; Make note of the username 22 and password chosen. 23 24The application assumes that both a development and production environment 25exist, therefore the above steps will need to be completed a second time. Now 26the following environment variables will need to be set in your application 27environment: 28 29* GCP_DB_INSTANCE_CONNECTION_NAME_DEV: The instance ID of the provisioned 30 database server; this is listed on the SQL homepage in Google Cloud 31 alongside the respective database instance 32* GCP_DB_NAME_DEV: the database name created from step 6 33* GCP_DB_USER_DEV: the username created from step 7 34* GCP_DB_PASSWORD_DEV: the password created from step 7 35* GCP_DB_PROXY_PORT_DEV: an arbitrary, unique port used for the local secure 36 MySQL proxy 37 38The same environment variables should be set for production. The names are the 39same but replace **"DEV"** with **"PROD"** 40 41One the environment variables are set, the database can be readied by running: 42 43`make db_upgrade` 44 45To run the same set of upgrades for production, run: 46 47`ROLE="prod" make db_upgrade` 48 49# Running the Application 50 51Executing the application will by default: 52 531. Clone all of the repositories specified in config.json and write the 54 observed differentials to the database 552. Run application-specified analytics on the persisted data and write the 56 output to designated denormalized tables; those tables are tightly coupled 57 to Datastudio data sources in the [AOSP Branch 58 Differentials](https://datastudio.google.com/c/u/0/org/UTgoe29uR0C3F1FBAYBSww/reporting/1lTzMXHBWiVvc0Dnb2DJvRQeTVCAIuvGF/page/9rqP) 593. Generate CSV reports based on persisted data 60 61Applications steps can be optionally disabled for presumably one-off runs: 62`ROLE="prod" ./repodiff 63[--execute-diff=false][--denormalize-data=false][--generate-report=false]` 64 65`config.json` will specify input and output parameters for the application; 66update to reflect your own environment. 67 68To run: 69 70`make run` 71 72For production: `ROLE="prod" make run` 73 74# Tooling 75 76To connect to the MySQL server used by the application, run: 77 78`make db_shell` 79 80To create new, canonical migration scripts to update the database schema, run: 81 82`make sql_script` 83 84To undo the latest database migration, run: 85 86`make db_downgrade` 87 88To upgrade to the latest database version, run: 89 90`make db_upgrade` 91 92To run tests, run: 93 94`make test` 95 96# Consuming the Application Output 97 98TLDR: Use [Google Data Studio](https://datastudio.google.com); Your datasource 99will be tables from the provisioned database set up in the provisioning 100instructions. All intended consumable tables are prefixed with 101**denormalized_view_** 102 103The rationale behind the application is that setup in Data Studio should require 104little to no learning curve (for developers especially), and views are simple 105projections of the underlying model. The application, then, should run whatever 106necessary logic to produce desired analytics that can be written to a 107denormalized database table for straightforward consumption. 108