To create the different jobs displayed in this article, you have to create a repository with saagie's platform information (value).
Query from Drill
Example: Count the number of lines.
- Create a new job
- Add the component tJDBCConnection : allows the creation of a JDBC database connection
- Add the component tJDBCInput : reads a DB table and extracts fields based on an SQL query
- Add the component tLogRow : displays results
- Create links:
- tJDBCConnection is connected with tJDBCInput (through "OnSubjobOk")
- tJDBCInput is connected with tLogRow (through "Main")
- Double click on tJDBCConnection and set its properties:
- Enter a URL JDBC (ex: jdbc:drill:drillbit=$IP_DRILL:$PORT_DRILL)
- Add a driver Jar.
- Download the jar : https://drill.apache.org/docs/using-the-jdbc-driver/ and use drill-jdbc-all-1.16.0.jar
- Enter a driver class. For Drill, the name of driver class is org.apache.drill.jdbc.Driver.
- Click on the tab Advanced settings and untick Use a auto-commit
- Double click on tJDBCInput and set its properties :
- Tick Use an existing connection
- Enter a name of table
- Enter your request : "SELECT COUNT(*) FROM " + context.Table_Name (ex Table_Name: `hive`.`opendata`.`prs_nat`)
- Run a job