* Using sqlite to illustrate models-as-data This section demonstrates the use of the models-as-data system by analyzing a small Java application that uses the SQLite JDBC driver. The example is adapted from a CodeQL workshop. ** Build the CodeQL Database To get started, build the CodeQL database for the SQLite-backed Java sample. Adjust paths as needed. #+BEGIN_SRC sh SRCDIR=$(pwd) DB=$SRCDIR/java-sqlite-$(cd $SRCDIR && git rev-parse --short HEAD).db echo $DB test -d "$DB" && rm -fR "$DB" mkdir -p "$DB" # Ensure the correct CodeQL version is in your PATH export PATH="$(cd ../codeql && pwd):$PATH" codeql database create --language=java -s . -j 8 -v $DB --command='./build.sh' # Check for presence of AddUser.java in the resulting database unzip -v $DB/src.zip | grep AddUser #+END_SRC Then add this database directory to your VS Code =DATABASES= tab. ** Tests Using a Default Query You can run the standard SQL injection query: [[../ql/java/ql/src/Security/CWE/CWE-089/SqlTainted.ql]] but it will return no results. However, it does help identify which classes are being analyzed as potential sources and sinks. Instead, run the diagnostic query: [[./Illustrations.ql]] You can run it from the CLI: #+BEGIN_SRC sh codeql query run \ -v \ --database java-sqlite-e2e555c.db \ --output result.bqrs \ --threads=12 \ --ram=14000 \ Illustrations.ql codeql bqrs decode --format=text result.bqrs | sed -n '/^Result set: #select/,$p' #+END_SRC The result will look like: #+BEGIN_SRC text Result set: #select | ui | qsi | +------+-------+ | args | query | #+END_SRC In the editor, these correspond to: 1. =main(String[] args)= — source-like 2. =conn.createStatement().executeUpdate(query)= — sink However, =System.console().readLine()= is not detected as a source. Therefore, =SqlTainted.ql= cannot find a complete flow. ** Supplement Sources via the Model Editor - [ ] We observe no flow from source to sink - A sink exists (=executeUpdate=) - But no recognized source is found - [ ] There are two ways to fix this: 1. Add a new source in =Customizations.qll= 2. Add a new source in the models-as-data YAML format ** Supplement CodeQL: Write a Full Manual Query A manual dataflow query is already available: [[./full-query.ql]] This can trace the data manually even when standard configuration fails. ** Supplement CodeQL: Add to FlowSource or a Subclass Sometimes, the only way to identify how to extend a source is to understand how CodeQL internally resolves source nodes. Key class hierarchies: #+BEGIN_SRC java abstract class SourceNode extends DataFlow::Node abstract class RemoteFlowSource extends SourceNode #+END_SRC Follow usage in: - [[../ql/java/ql/lib/Customizations.qll]] - [[../ql/java/ql/src/Security/CWE/CWE-089/SqlTainted.ql]] Then modify =Customizations.qll= by adding the custom source. The modified [[../ql/java/ql/lib/Customizations.qll]] is #+BEGIN_SRC java import java private import semmle.code.java.dataflow.FlowSources class ReadLine extends RemoteFlowSource { ReadLine() { exists(Call read | read.getCallee().getName() = "readLine" and read = this.asExpr() ) } override string getSourceType() { result = "Console readline" } } #+END_SRC This allows #+BEGIN_SRC java predicate isSource(DataFlow::Node src) { src instanceof ActiveThreatModelSource } #+END_SRC to include =readLine()= even though we extended =RemoteFlowSource=. ** TODO supplement codeql: Add to models-as-data - schema in codeql: [[../ql/java/ql/lib/semmle/code/java/dataflow/internal/ExternalFlowExtensions.qll]] - data sample: [[../.github/codeql/extensions/jedis-db-local-java/models/redis.clients.jedis.model.yml]] In the model editor, we see a java.io.*Console.*readline' (using =show already modeled= option) #+BEGIN_SRC sh 1:$ rg -i 'java.io.*Console.*readline' ql/java ql/java/ql/lib/ext/generated/java.io.model.yml 16: - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] 17: - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[0]", "Argument[this]", "taint", "df-generated"] 18: - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[1].ArrayElement", "Argument[this]", "taint", "df-generated"] 19: - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] #+END_SRC note: this file is in the generated/ tree. There are others. The current readline modeling is in the =summaryModel= section; we need it in a =sourceModel= #+BEGIN_SRC yaml extensions: - addsTo: pack: codeql/java-all extensible: summaryModel data: ... - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[0]", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[1].ArrayElement", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument #+END_SRC The model editor will not show this because its already modeled. To illustrate text-based additions, we'll use plain text. Starting from #+BEGIN_SRC yaml extensions: - addsTo: pack: codeql/java-all extensible: summaryModel data: ... - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[0]", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[1].ArrayElement", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument #+END_SRC and the field information #+BEGIN_SRC java extensible predicate sourceModel( string package, string type, boolean subtypes, string name, string signature, string ext, string output, string kind, string provenance, QlBuiltins::ExtensionId madId ); #+END_SRC Starting from =summaryModel= #+BEGIN_SRC yaml # summaryModel # string package, string type, boolean subtypes, string name, string signature, string ext, string input, string output, string kind, string provenance, QlBuiltins::ExtensionId madId - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] #+END_SRC we can construct the =sourceModel= #+BEGIN_SRC yaml extensions: - addsTo: pack: codeql/java-all extensible: sourceModel data: # sourceModel # string package, string type, boolean subtypes, string name, string signature, string ext, string output, string kind, string provenance, QlBuiltins::ExtensionId madId - ["java.io", "Console", False, "readLine", "()", "", "ReturnValue", "remote", "manual"] # # from original # # summaryModel # # string package, string type, boolean subtypes, string name, string signature, string ext, string input, string output, string kind, string provenance, QlBuiltins::ExtensionId madId # - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] #+END_SRC and move this into [[../.github/codeql/extensions/sqlite-db/models/sqlite.model.yml]] To ensure that these model extensions are applied during query runs, include this setting #+begin_src javascript { ..., "settings": { ..., "codeQL.runningQueries.useExtensionPacks": "all" } } #+end_src in the workspace configuration file [[../qllab.code-workspace]] In some environments (e.g., older VS Code versions), you may also need to replicate this setting in [[../.vscode/settings.json]]; there it simplifies to #+begin_src javascript "codeQL.runningQueries.useExtensionPacks": "all" #+end_src Now we can run [[../ql/java/ql/src/Security/CWE/CWE-089/SqlTainted.ql]] again.