Whale exposes a direct line into
SQLAlchemy against connections defined in
~/.whale/config/connections.yaml through the
wh run command.
wh run filename.sql -w warehouse_name
-w flag is not given, the first warehouse in your
connections.yaml file will be used.
If there are multiple warehouses with the same
warehouse_name the credentials from the first matching
warehouse_name encountered will be used.
Note: this only works for (a) direct connections to warehouses (not the Hive metastore) and (b) connections where permissions allow for query runs.
wh run also supports Jinja2 templating -- for more information on how to set this up, see Jinja2 templating.
If you'd like to store your results automatically as comments within your
.sql file, you can add
--!wh-run to any SQL file (on its own line) and any
wh run execution will automatically string replace this line with the results of the query execution.
For example, the following sql file
select * from census.newtable limit 1--!wh-run
would, upon execution using
wh run, would be overwritten as:
select * from census.newtable limit 1/* results: 2021-01-12 14:42:24.318297--------------------------------------customer_id date1 test_field0 1 2019-10-01 None*/
wh run command can also be used against files with the
.markdown extension, to enable more advanced Jupyter-notebook style behavior within your IDE.
Any block (```sql) containing
--!wh-run will flag said block will:
Flag this block for execution when
wh run is called on this file.
Upon execution, replace the line containing
--!wh-run with a block of results.
This behavior is very similar to the execution within SQL files as shown above, but enables execution of SQL query blocks alongside context written in Markdown.
For instance, the following Markdown file
# My projectI've found that we've had some problems with this census table!```sqlselect * from census.newtable limit 1--!wh-run```
would be overwritten upon execution of
wh run as:
# My projectI've found that we've had some problems with this census table!```sqlselect * from census.newtable limit 1/* results: 2021-01-12 14:42:24.318297--------------------------------------customer_id date1 test_field0 1 2019-10-01 None*/```
For those familiar with Jupyter or R markdown notebooks, this pattern might feel somewhat foreign. Unlike with full-fledged programming languages, there is no need for logic to be transferred between cells in SQL, so we've made the conscious decision to simplify the usage pattern and supporting code by not building out support for execution of individual code blocks except for through explicit inclusion of the
--!wh-run statement. This has the added benefit of being minimally destructive, only replacing the
--!wh-run character sequence.
Add the following to your
.vimrc file, and you can run
<Leader>wh to execute whale files.
<Leader> is generally
nmap <Leader>wh :w<CR>:exec '!~/.whale/bin/whale run %'<CR>:e<CR>