* Using sqlite to illustrate models-as-data This description uses / recycles a codeql workshop. ** Build the codeql database To get started, build the codeql database (adjust paths to your setup): #+BEGIN_SRC sh # Build the db with source commit id. SRCDIR=$(pwd) DB=$SRCDIR/java-sqlite-$(cd $SRCDIR && git rev-parse --short HEAD).db echo $DB test -d "$DB" && rm -fR "$DB" mkdir -p "$DB" # Use the correct codeql export PATH="$(cd ../codeql && pwd):$PATH" codeql database create --language=java -s . -j 8 -v $DB --command='./build.sh' # Check for AddUser in the db unzip -v $DB/src.zip | grep AddUser #+END_SRC Then add this database directory to your VS Code =DATABASES= tab. ** Tests using a default query You can run the stdlib query [[../ql/java/ql/src/Security/CWE/CWE-089/SqlTainted.ql]] but will get no results. It does point at classes to inspect -- in particular, the source and sink classes. Run [[./Illustrations.ql]]; from the command line or vs studio code. Via cli: #+BEGIN_SRC sh # run query codeql query run \ -v \ --database java-sqlite-e2e555c.db \ --output result.bqrs \ --threads=12 \ --ram=14000 \ Illustrations.ql # format results codeql bqrs decode --format=text result.bqrs | sed -n '/^Result set: #select/,$p' #+END_SRC This shows #+BEGIN_SRC text Result set: #select | ui | qsi | +------+-------+ | args | query | #+END_SRC In the editor, these link to 1. =main(ARGS)= and 2. =conn.createStatement().executeUpdate(QUERY);= The second is correct, but =System.console().readLine();= is not found. Thus, =SqlTainted.ql= will not find anything. ** TODO supplement sources via the model editor - [ ] We have no flow + check source, sink + we have a sink + but ActiveThreatModelSource finds no source - [ ] We can supplement in different ways ** supplement codeql: Write full manual query: already in workshop ** supplement codeql: Add to FlowSource or a subclass Note: this /one area/ that just has to be known. Browsing source will *not* help you. CodeQL reading hint: : class ActiveThreatModelSource extends DataFlow::Node uses : this.(SourceNode).getThreatModel() So following the cast (SourceNode) may be useful: #+BEGIN_SRC java /** ,* A data flow source. ,*/ abstract class SourceNode extends DataFlow::Node #+END_SRC Following the =abstract class= is promising: #+BEGIN_SRC java abstract class RemoteFlowSource extends SourceNode #+END_SRC and others. In [[../ql/java/ql/lib/Customizations.qll]] notice the comments mentioning RemoteFlowSource. Use imports from [[../ql/java/ql/src/Security/CWE/CWE-089/SqlTainted.ql]] but note that there are conflicts. you will use : private import semmle.code.java.dataflow.FlowSources Follow this to FlowSources, and find the mentioned RemoteFlowSource : abstract class RemoteFlowSource extends SourceNode Add the custom source. The modified [[../ql/java/ql/lib/Customizations.qll]] is #+BEGIN_SRC java import java private import semmle.code.java.dataflow.FlowSources class ReadLine extends RemoteFlowSource { ReadLine() { exists(Call read | read.getCallee().getName() = "readLine" and read = this.asExpr() ) } override string getSourceType() { result = "Console readline" } } #+END_SRC Note that the predicate #+BEGIN_SRC java module QueryInjectionFlowConfig implements DataFlow::ConfigSig { predicate isSource(DataFlow::Node src) { src instanceof ActiveThreatModelSource } ...; } #+END_SRC now also returns the readLine() result -- although we extended RemoteFlowSource, not ActiveThreatModelSource ** supplement codeql: Add to models-as-data - schema in codeql: [[../ql/java/ql/lib/semmle/code/java/dataflow/internal/ExternalFlowExtensions.qll]] - data sample: [[../.github/codeql/extensions/jedis-db-local-java/models/redis.clients.jedis.model.yml]] In the model editor, we see a java.io.*Console.*readline' (using =show already modeled= option) #+BEGIN_SRC sh 1:$ rg -i 'java.io.*Console.*readline' ql/java ql/java/ql/lib/ext/generated/java.io.model.yml 16: - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] 17: - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[0]", "Argument[this]", "taint", "df-generated"] 18: - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[1].ArrayElement", "Argument[this]", "taint", "df-generated"] 19: - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] #+END_SRC note: this file is in the generated/ tree. The current readline modeling is in the =summaryModel= section; we need it in a =sourceModel= #+BEGIN_SRC yaml extensions: - addsTo: pack: codeql/java-all extensible: summaryModel data: ... - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[0]", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[1].ArrayElement", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument #+END_SRC The model editor will not show this because its already modeled. To illustrate text-based additions, we'll use plain text. Starting from #+BEGIN_SRC yaml extensions: - addsTo: pack: codeql/java-all extensible: summaryModel data: ... - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[0]", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument[1].ArrayElement", "Argument[this]", "taint", "df-generated"] - ["java.io", "Console", False, "readLine", "(String,Object[])", "", "Argument #+END_SRC and the field information #+BEGIN_SRC java extensible predicate sourceModel( string package, string type, boolean subtypes, string name, string signature, string ext, string output, string kind, string provenance, QlBuiltins::ExtensionId madId ); #+END_SRC Starting from =summaryModel= #+BEGIN_SRC yaml # summaryModel # string package, string type, boolean subtypes, string name, string signature, string ext, string input, string output, string kind, string provenance, QlBuiltins::ExtensionId madId - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] #+END_SRC we can construct the =sourceModel= #+BEGIN_SRC yaml extensions: - addsTo: pack: codeql/java-all extensible: sourceModel data: # sourceModel # string package, string type, boolean subtypes, string name, string signature, string ext, string output, string kind, string provenance, QlBuiltins::ExtensionId madId - ["java.io", "Console", False, "readLine", "()", "", "ReturnValue", "remote", "manual"] # # from original # # summaryModel # # string package, string type, boolean subtypes, string name, string signature, string ext, string input, string output, string kind, string provenance, QlBuiltins::ExtensionId madId # - ["java.io", "Console", False, "readLine", "()", "", "Argument[this]", "ReturnValue", "taint", "df-generated"] #+END_SRC and move this into [[../.github/codeql/extensions/sqlite-db/models/sqlite.model.yml]] To ensure that these model extensions are applied during query runs, include this setting #+begin_src javascript { ..., "settings": { ..., "codeQL.runningQueries.useExtensionPacks": "all" } } #+end_src in the workspace configuration file [[../qllab.code-workspace]] In some environments (e.g., older VS Code versions), you may also need to replicate this setting in [[../.vscode/settings.json]]; there it simplifies to #+begin_src javascript "codeQL.runningQueries.useExtensionPacks": "all" #+end_src Now we can run [[../ql/java/ql/src/Security/CWE/CWE-089/SqlTainted.ql]] again.