Commit Graph

18064 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
9ed15732ed Python: Update set literal formatting
Now that auto-formatting rules have been updated
2020-11-11 09:38:25 +01:00
Rasmus Lerchedahl Petersen
0710963fc3 Python: update test expectations
EssaNode -> ControlFlowNode
2020-11-10 23:58:55 +01:00
Erik Krogh Kristensen
e01d4b104e update expected output 2020-11-10 23:42:38 +01:00
Mathias Vorreiter Pedersen
8d4b948acf Merge branch 'main' into interleave-op-instr-field-flow 2020-11-10 21:15:50 +01:00
Erik Krogh Kristensen
6b7f8dd505 fix parse errors related to char escapes and char ranges 2020-11-10 21:02:29 +01:00
Robert Marsh
14f1fa50f1 Merge branch 'main' into rdmarsh2/cpp/remove-initialize-nonlocal
Accept test changes from IR temporaries and block ordering
2020-11-10 11:14:26 -08:00
Jonas Jensen
fc764db8e1 Merge pull request #4643 from nickrolfe/getFileBySourceArchiveName
Replace getEncodedFile with shared getFileBySourceArchiveName predicate
2020-11-10 17:36:29 +01:00
Mathias Vorreiter Pedersen
884554692f Merge pull request #4645 from geoffw0/oddsends2
C++: Odds and ends
2020-11-10 16:59:10 +01:00
Mathias Vorreiter Pedersen
618d9c33a1 Merge pull request #4642 from geoffw0/memberfn
C++: Improve MemberFunction.qll data flows
2020-11-10 16:54:42 +01:00
Jonas Jensen
da90717ef0 Merge pull request #4641 from aschackmull/autoformat-setliterals
C++/C#/Java/JavaScript/Python: Autoformat set literals.
2020-11-10 16:11:11 +01:00
Tom Hvitved
7e89e57f5d Merge pull request #4606 from hvitved/csharp/dataflow/ef
C#: Precise data flow for EntityFramework(Core)
2020-11-10 15:54:20 +01:00
Nick Rolfe
ac4a1f1d9b Update comment to be a QLDoc comment 2020-11-10 14:14:27 +00:00
Geoffrey White
be537a8021 C++: Move ThrowExpr, ReThrowExpr definitions to a more logical location. 2020-11-10 14:06:58 +00:00
Nick Rolfe
1e1eb7ee33 Replace getEncodedFile with shared getFileBySourceArchiveName predicate
While also making it work with paths for databases created on Windows.
2020-11-10 13:55:27 +00:00
Geoffrey White
2beb7824ec C++: Backticks. 2020-11-10 13:35:05 +00:00
Geoffrey White
599034668f C++: Fix typo. 2020-11-10 13:32:31 +00:00
Geoffrey White
f109e8ccbc C++: Remove duplicate import. 2020-11-10 13:31:47 +00:00
Geoffrey White
c4fe8df04c C++: Support data flow from parameter deref to return value. 2020-11-10 13:17:34 +00:00
Geoffrey White
4563e32e92 C++: Remove data flows we shouldn't need in MemberFunction. 2020-11-10 13:17:24 +00:00
Anders Schack-Mulligen
89ef6ea4eb C++/C#/Java/JavaScript/Python: Autoformat set literals. 2020-11-10 13:32:27 +01:00
Alvaro Muñoz
02cf49a773 apply codeql formatting 2020-11-10 11:46:42 +01:00
Rasmus Lerchedahl Petersen
109d55eb25 Python: Make ParameterNode a CfgNode
Add a step from that `CfgNode` to the corresponding `EssaNode`.
The intended effect is seen in `ImpliesDataflow.expected`.
The efeect seen in other `.expected`-files is that parameter nodes
change type, that the extra steps are seen, and that flow from
`EssaVar`s is mirrored in flow from `CfgNode`s.
There is one surprise, which is the `.0` node in
`coverage/localFlow.expected`.
2020-11-10 11:35:50 +01:00
Alvaro Muñoz
24a47fbb0f additional qldoc commentes 2020-11-10 10:48:47 +01:00
Alvaro Muñoz
3545edb92c address code review suggestions 2020-11-10 10:45:14 +01:00
Tom Hvitved
697b0ba4c1 C#: Address review comments 2020-11-10 10:42:05 +01:00
Erik Krogh Kristensen
ae7c7607f1 fix typos in documentation strings
Co-authored-by: intrigus-lgtm <60750685+intrigus-lgtm@users.noreply.github.com>
2020-11-10 10:41:39 +01:00
Erik Krogh Kristensen
b5b4d6d62a add change note 2020-11-10 10:41:39 +01:00
Erik Krogh Kristensen
5ecae55e77 add keys used by jsonwebtoken as CredentialsExpr 2020-11-10 10:41:39 +01:00
Erik Krogh Kristensen
e75259d3a6 model the verify function in jsonwebtoken 2020-11-10 10:41:39 +01:00
Erik Krogh Kristensen
6732493377 add model for jwt-decode 2020-11-10 10:41:36 +01:00
yoff
26286e534e Merge pull request #4174 from yoff/SharedDataflow_PointsToImpliesDataflow
Python: Dataflow, Test that `pointsTo` implies data flow
merging now, will fix `self` in a later PR
2020-11-10 10:25:29 +01:00
Robert Marsh
c00587d2cb C++/C#: Conflated memory as IR dump annotation
Removes the IR consistency checks for conflated memory and marks
instructions that have a conflated result with a percent sign (%)
instead. This avoids reimplementing part of the alias analysis logic
in the consistency check.
2020-11-09 14:55:47 -08:00
CodeQL CI
1e048d8045 Merge pull request #4609 from asgerf/js/destructuring-export
Approved by erik-krogh
2020-11-09 15:47:00 +00:00
Anders Schack-Mulligen
31ec79819e Merge pull request #4631 from luchua-bc/java-nfe-library
Java: Factor NumberFormatException out into a library file
2020-11-09 13:50:31 +01:00
yoff
3215f50b73 Merge pull request #4632 from RasmusWL/python-move-configurations-out-of-queries
Python: move configurations out of queries
2020-11-09 13:10:33 +01:00
luchua-bc
d765c7bbb2 Update qldoc 2020-11-09 11:23:48 +00:00
Asger Feldthaus
32c5bc4836 JS: Update test case output (new flow is correct) 2020-11-09 11:19:47 +00:00
Mathias Vorreiter Pedersen
25ba6ca160 Merge pull request #4637 from jbj/temp-objects-perf-fixup
C++: pragma[noinline] parameter index predicates
2020-11-09 10:12:50 +01:00
Mathias Vorreiter Pedersen
3c2fb5a93f Merge branch 'main' into interleave-op-instr-field-flow 2020-11-09 09:58:19 +01:00
Rasmus Wriedt Larsen
fbe51c51bb Python: Add missing QLDoc 2020-11-09 09:05:08 +01:00
Jonas Jensen
b8b1f0c617 C++: pragma[noinline] parameter index predicates
A performance regression in `definitionByReferenceNodeFromArgument#ff`
was ultimately caused by a join on parameter indexes in
`DefinitionByReferenceNode.getArgument`. Joining on numbers in QL is
always fragile, and somehow the changes in #4432 had caused the join
order here to break.

Instead of tweaking the join order in the slow predicate itself, I added
`pragma[noinline]` to one of the predicates involved in the join on
parameter indexes. This should prevent us from getting similar
performance problems in the future when we write code that joins on
parameter numbers. Joining on indexes is always risky, but it's even
more risky when one of the predicates in the join is inlined by the
compiler and expands to further joins.

I tested performance by running `CgiXss.ql` on a ChakraCore snapshot.
Tuple counts before (I interrupted execution after five minutes or so):

    (626s) Tuple counts for DataFlowUtil::definitionByReferenceNodeFromArgument#ff:
    58162      ~0%     {3} r1 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, -1, I.<0>
    26934      ~0%     {2} r2 = JOIN r1 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 2 OUTPUT r1.<0>, r1.<2>
    26934      ~1%     {2} r3 = JOIN r2 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r2.<1>
    26850      ~1%     {2} r4 = JOIN r3 WITH Instruction::CallInstruction::getThisArgumentOperand_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r3.<1>
    26850      ~0%     {2} r5 = JOIN r4 WITH Operand::Operand::getDef_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r4.<1>
    26850      ~1%     {2} r6 = JOIN r5 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r5.<1>
    58162      ~0%     {2} r7 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, I.<0>
    58162      ~4%     {3} r8 = JOIN r7 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 1 OUTPUT R.<1>, r7.<1>, r7.<0>
    4026581120 ~0%     {4} r9 = JOIN r8 WITH Instruction::CallInstruction::getPositionalArgumentOperand_dispred#fff_102#join_rhs AS R ON FIRST 1 OUTPUT r8.<2>, R.<1>, r8.<1>, R.<2>
    31154      ~4%     {2} r10 = JOIN r9 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 2 OUTPUT r9.<3>, r9.<2>
    31154      ~8%     {2} r11 = JOIN r10 WITH Operand::Operand::getDef_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r10.<1>
    31154      ~0%     {2} r12 = JOIN r11 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r11.<1>
    58004      ~0%     {2} r13 = r6 \/ r12
                       return r13

Tuple counts after:

    (0s) Tuple counts for DataFlowUtil::definitionByReferenceNodeFromArgument#ff:
    385785  ~6%     {2} r1 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, I.<0>
    385785  ~0%     {3} r2 = JOIN r1 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 1 OUTPUT r1.<0>, r1.<1>, R.<1>
    385785  ~1%     {3} r3 = JOIN r2 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r2.<2>, r2.<1>
    198736  ~4%     {2} r4 = JOIN r3 WITH Instruction::CallInstruction::getPositionalArgument#fff AS R ON FIRST 2 OUTPUT R.<2>, r3.<2>
    198736  ~0%     {2} r5 = JOIN r4 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r4.<1>
    385785  ~1%     {3} r6 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, -1, I.<0>
    186891  ~1%     {2} r7 = JOIN r6 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 2 OUTPUT r6.<0>, r6.<2>
    186891  ~2%     {2} r8 = JOIN r7 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r7.<1>
    183201  ~3%     {2} r9 = JOIN r8 WITH Instruction::CallInstruction::getThisArgumentOperand_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r8.<1>
    183201  ~0%     {2} r10 = JOIN r9 WITH Operand::Operand::getDef_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r9.<1>
    175449  ~8%     {2} r11 = JOIN r10 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r10.<1>
    374185  ~3%     {2} r12 = r5 \/ r11
                    return r12
2020-11-09 09:01:22 +01:00
Rasmus Wriedt Larsen
ed0e4f8425 Python: reasoning about => detecting
Co-authored-by: yoff <lerchedahl@gmail.com>
2020-11-09 09:01:04 +01:00
Aditya Sharad
e75082a249 Merge pull request #4620 from github/codeql-docs-reorg-1
[docs] Rename source files to match article titles
2020-11-06 12:18:14 -08:00
luchua-bc
d568eb635f Update qldoc 2020-11-06 15:33:26 +00:00
Geoffrey White
0790fb6324 Update cpp/change-notes/2020-11-02-unused-local-variable.md
Co-authored-by: Jonas Jensen <jbj@github.com>
2020-11-06 14:42:48 +00:00
yoff
45317bcec9 Update python/ql/test/library-tests/PointsTo/new/code/w_function_values.py
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2020-11-06 15:03:20 +01:00
Rasmus Wriedt Larsen
9ebe59d393 Python: Move UnsafeDeserialization configuration to own file 2020-11-06 14:27:37 +01:00
luchua-bc
450ff26694 Convert the query to a library 2020-11-06 13:25:00 +00:00
Rasmus Wriedt Larsen
d38c48d2c8 Python: Move ReflectedXSS configuration to own file 2020-11-06 14:24:31 +01:00
Rasmus Wriedt Larsen
1897a0d59a Python: Move PathInjection configuration to own file
This one required a bit more thought, but ended up pretty nicely. Had to write
some QLDoc, but I think it turned out OK.
2020-11-06 14:21:23 +01:00