Commit Graph

17823 Commits

Author SHA1 Message Date
Mathias Vorreiter Pedersen
a330cae2f9 C++: Update change-notes. 2020-11-10 17:24:45 +01:00
Mathias Vorreiter Pedersen
4bcf1f498b C++: Add new query for unsafe use of this. 2020-11-10 17:23:55 +01:00
Mathias Vorreiter Pedersen
25ba6ca160 Merge pull request #4637 from jbj/temp-objects-perf-fixup
C++: pragma[noinline] parameter index predicates
2020-11-09 10:12:50 +01:00
Jonas Jensen
b8b1f0c617 C++: pragma[noinline] parameter index predicates
A performance regression in `definitionByReferenceNodeFromArgument#ff`
was ultimately caused by a join on parameter indexes in
`DefinitionByReferenceNode.getArgument`. Joining on numbers in QL is
always fragile, and somehow the changes in #4432 had caused the join
order here to break.

Instead of tweaking the join order in the slow predicate itself, I added
`pragma[noinline]` to one of the predicates involved in the join on
parameter indexes. This should prevent us from getting similar
performance problems in the future when we write code that joins on
parameter numbers. Joining on indexes is always risky, but it's even
more risky when one of the predicates in the join is inlined by the
compiler and expands to further joins.

I tested performance by running `CgiXss.ql` on a ChakraCore snapshot.
Tuple counts before (I interrupted execution after five minutes or so):

    (626s) Tuple counts for DataFlowUtil::definitionByReferenceNodeFromArgument#ff:
    58162      ~0%     {3} r1 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, -1, I.<0>
    26934      ~0%     {2} r2 = JOIN r1 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 2 OUTPUT r1.<0>, r1.<2>
    26934      ~1%     {2} r3 = JOIN r2 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r2.<1>
    26850      ~1%     {2} r4 = JOIN r3 WITH Instruction::CallInstruction::getThisArgumentOperand_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r3.<1>
    26850      ~0%     {2} r5 = JOIN r4 WITH Operand::Operand::getDef_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r4.<1>
    26850      ~1%     {2} r6 = JOIN r5 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r5.<1>
    58162      ~0%     {2} r7 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, I.<0>
    58162      ~4%     {3} r8 = JOIN r7 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 1 OUTPUT R.<1>, r7.<1>, r7.<0>
    4026581120 ~0%     {4} r9 = JOIN r8 WITH Instruction::CallInstruction::getPositionalArgumentOperand_dispred#fff_102#join_rhs AS R ON FIRST 1 OUTPUT r8.<2>, R.<1>, r8.<1>, R.<2>
    31154      ~4%     {2} r10 = JOIN r9 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 2 OUTPUT r9.<3>, r9.<2>
    31154      ~8%     {2} r11 = JOIN r10 WITH Operand::Operand::getDef_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r10.<1>
    31154      ~0%     {2} r12 = JOIN r11 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r11.<1>
    58004      ~0%     {2} r13 = r6 \/ r12
                       return r13

Tuple counts after:

    (0s) Tuple counts for DataFlowUtil::definitionByReferenceNodeFromArgument#ff:
    385785  ~6%     {2} r1 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, I.<0>
    385785  ~0%     {3} r2 = JOIN r1 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 1 OUTPUT r1.<0>, r1.<1>, R.<1>
    385785  ~1%     {3} r3 = JOIN r2 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r2.<2>, r2.<1>
    198736  ~4%     {2} r4 = JOIN r3 WITH Instruction::CallInstruction::getPositionalArgument#fff AS R ON FIRST 2 OUTPUT R.<2>, r3.<2>
    198736  ~0%     {2} r5 = JOIN r4 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r4.<1>
    385785  ~1%     {3} r6 = SCAN DataFlowUtil::DefinitionByReferenceNode#class#ff AS I OUTPUT I.<1>, -1, I.<0>
    186891  ~1%     {2} r7 = JOIN r6 WITH Instruction::IndexedInstruction#ff AS R ON FIRST 2 OUTPUT r6.<0>, r6.<2>
    186891  ~2%     {2} r8 = JOIN r7 WITH Instruction::SideEffectInstruction::getPrimaryInstruction_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r7.<1>
    183201  ~3%     {2} r9 = JOIN r8 WITH Instruction::CallInstruction::getThisArgumentOperand_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r8.<1>
    183201  ~0%     {2} r10 = JOIN r9 WITH Operand::Operand::getDef_dispred#3#ff AS R ON FIRST 1 OUTPUT R.<1>, r9.<1>
    175449  ~8%     {2} r11 = JOIN r10 WITH Instruction::Instruction::getUnconvertedResultExpression_dispred#ff AS R ON FIRST 1 OUTPUT R.<1>, r10.<1>
    374185  ~3%     {2} r12 = r5 \/ r11
                    return r12
2020-11-09 09:01:22 +01:00
CodeQL CI
9f2eb84f2b Merge pull request #4624 from erik-krogh/concatFix
Approved by asgerf
2020-11-06 09:11:41 +00:00
Anders Schack-Mulligen
cb77e460ae Merge pull request #4600 from porcupineyhairs/urirefactor
Java : Refactor all instances of `java.net.URI` into TypeUri
2020-11-06 09:35:09 +01:00
CodeQL CI
a908e5938e Merge pull request #4574 from erik-krogh/jsdom
Approved by asgerf
2020-11-05 22:13:39 +00:00
Erik Krogh Kristensen
9137759d7c calculate the size of the concatenation before doing the actual concatenation in Expr.qll 2020-11-05 22:55:52 +01:00
Tom Hvitved
a3894be1c5 Merge pull request #4607 from hvitved/csharp/msbuild-mono-no-shared-compilation
C#: Disable shared compilation when building with Mono+MSBuild
2020-11-05 19:56:25 +01:00
Erik Krogh Kristensen
e124ba66b4 moving jsdom sink to js/xss 2020-11-05 16:10:33 +01:00
Tom Hvitved
10ab3304c1 Merge pull request #4575 from hvitved/csharp/cfg/post-dominance
C#: Restrict post-dominance to normal execution
2020-11-05 15:31:13 +01:00
Anders Schack-Mulligen
45d117b68e Merge pull request #4603 from pwntester/new_deser_sink
New UnsafeDeserialization sink and improvements to SnakeYaml sink
2020-11-05 13:09:15 +01:00
Alvaro Muñoz
f103955f38 change qldoc formating according to LSP suggestion 2020-11-05 11:48:26 +01:00
CodeQL CI
89a808cafe Merge pull request #4552 from erik-krogh/tsImport
Approved by asgerf
2020-11-05 09:23:58 +00:00
CodeQL CI
b55f18bffd Merge pull request #4549 from erik-krogh/pruneReturn
Approved by asgerf
2020-11-05 09:13:21 +00:00
CodeQL CI
c85f817cee Merge pull request #4579 from erik-krogh/redos
Approved by asgerf
2020-11-05 08:38:44 +00:00
Robert Marsh
2f204869e7 Merge pull request #4604 from criemen/ir-block-sort-order
C++, C# IR: Stabilize sort order for basic blocks.
2020-11-04 18:22:23 -05:00
Erik Krogh Kristensen
342b6a4f2d Update javascript/ql/src/semmle/javascript/security/performance/SuperlinearBackTracking.qll
Co-authored-by: Asger F <asgerf@github.com>
2020-11-04 22:37:56 +01:00
Alvaro Muñoz
302062b670 Merge branch 'new_deser_sink' of https://github.com/pwntester/ql into new_deser_sink 2020-11-04 18:58:57 +01:00
Alvaro Muñoz
6fef63306e add qldoc 2020-11-04 18:58:41 +01:00
Erik Krogh Kristensen
e16fa0668a update expected output 2020-11-04 18:24:31 +01:00
Geoffrey White
48628fa195 Merge pull request #4589 from criemen/model-vector-emplace
C++: Model std::vector emplace and emplace_back()
2020-11-04 16:09:31 +00:00
Tom Hvitved
b5063bbcb5 Merge pull request #4495 from hvitved/csharp/dataflow/summaries
C#: Shared interface/implementation for flow summaries
2020-11-04 17:02:19 +01:00
Cornelius Riemenschneider
78d885ee7d C++: Accept test output. 2020-11-04 16:45:07 +01:00
Cornelius Riemenschneider
44d6584fa2 C++, C#: Auto-format. 2020-11-04 16:26:56 +01:00
Cornelius Riemenschneider
a13947424a C++, C# IR: Stabilize sort order for basic blocks. 2020-11-04 16:26:56 +01:00
Cornelius Riemenschneider
e7e5754270 C++: Add taint model for std::vector::emplace/_back. 2020-11-04 16:20:01 +01:00
Erik Krogh Kristensen
03c46c9be0 autoformat 2020-11-04 16:18:24 +01:00
yoff
79fcf598f3 Merge pull request #4608 from RasmusWL/patch-1
Python: Remove unnecessary cached annotation from adjacentRefUse
2020-11-04 16:08:30 +01:00
Cornelius Riemenschneider
62a02cde73 C++: Add test to show missing taint steps for std::vector::emplace/_back 2020-11-04 16:07:15 +01:00
Cornelius Riemenschneider
081ab1e2bb C++: Add std::vector::emplace/_back to our STL model. 2020-11-04 16:07:08 +01:00
Ian Lynagh
6ff939db5c Merge pull request #4432 from dbartol/dbartol/temporaries/work
C++: Represent temporary object initialization in AST and IR
2020-11-04 14:38:45 +00:00
Rasmus Wriedt Larsen
31247739d7 Python: Remove unnecessary cached annotation from adjacentRefUse
As discussed in https://github.com/github/codeql/pull/4544#pullrequestreview-516575676
2020-11-04 15:16:08 +01:00
yoff
62cb4ec974 Merge pull request #4605 from RasmusWL/python-fix-django-response-modeling
Python: fix django response modeling
2020-11-04 15:00:52 +01:00
Tom Hvitved
131a05563a C#: Disable shared compilation when building with Mono+MSBuild 2020-11-04 14:16:25 +01:00
Porcupiney Hairs
0a028dcb47 Java : Refactor all instances of java.net.URI into TypeUri 2020-11-04 18:23:26 +05:30
Rasmus Wriedt Larsen
5cf8285717 Python: Fix default mimetype for django FileResponse 2020-11-04 12:28:51 +01:00
Rasmus Wriedt Larsen
826aedeb85 Python: Remove resolved TODO 2020-11-04 12:17:31 +01:00
Rasmus Wriedt Larsen
353505ec6c Python: Handle content of Django redirects correctly 2020-11-04 12:10:58 +01:00
Taus
180373c41d Merge pull request #4597 from yoff/python-fix-ql-doc
Python: Fix ql doc
2020-11-04 11:37:32 +01:00
Rasmus Wriedt Larsen
92dc7dc2f3 Python: Use mimetype instead of content-type in django modeling
This enables the XSS query to actually find results from django responses.
2020-11-04 11:34:20 +01:00
Alvaro Muñoz
aa7b87aa33 Update java/change-notes/2020-11-04-commonslang-unsafe-deserialization-sinks.md
Co-authored-by: Anders Schack-Mulligen <aschackmull@users.noreply.github.com>
2020-11-04 10:58:27 +01:00
Alvaro Muñoz
b284141a16 Merge branch 'new_deser_sink' of https://github.com/pwntester/ql into new_deser_sink 2020-11-04 10:51:07 +01:00
Alvaro Muñoz
436563d914 ChangeNote for new unsafe deserialization sinks 2020-11-04 10:50:50 +01:00
Anders Schack-Mulligen
22b4df0f3c Merge pull request #4512 from luchua-bc/sensitive-broadcast
Java: Sensitive broadcast
2020-11-04 10:47:48 +01:00
Alvaro Muñoz
6f78b725e6 Apply suggestions from code review
Co-authored-by: Anders Schack-Mulligen <aschackmull@users.noreply.github.com>
2020-11-04 10:43:37 +01:00
Anders Schack-Mulligen
26495225e0 Update java/ql/src/experimental/Security/CWE/CWE-927/SensitiveBroadcast.qhelp
Co-authored-by: Marcono1234 <Marcono1234@users.noreply.github.com>
2020-11-04 10:05:55 +01:00
Tom Hvitved
c5abf29dfc C#: Update flow-summary test 2020-11-03 20:28:42 +01:00
Dave Bartolomeo
4cc9110dbd Fix test expectation 2020-11-03 13:39:33 -05:00
Dave Bartolomeo
f0b9794907 Merge remote-tracking branch 'upstream/main' into work 2020-11-03 11:33:44 -05:00