Commit Graph

130 Commits

Author SHA1 Message Date
Rasmus Wriedt Larsen
a81d359669 Python: Model marshal.load 2021-10-07 21:27:51 +02:00
Rasmus Wriedt Larsen
1b61296ea5 Python: Model pickle.load 2021-10-07 21:25:48 +02:00
Rasmus Wriedt Larsen
27c368a444 Python: Model keyword arguments to pickle.loads 2021-10-07 21:24:12 +02:00
Rasmus Wriedt Larsen
a31bf75169 Python: Refactor pickle.loads() modeling 2021-10-07 20:28:30 +02:00
Anders Schack-Mulligen
2b88a2aa0c Dataflow: Fix qldoc: s/accesspath/access path/. 2021-10-07 14:46:24 +02:00
Taus
54aec7bb96 Python: Fix bad magic in controls/2
The changes to `ModificationOfParameterWithDefault.ql` and the use of
`ConditionBlock::controls` therein caused the `BasicBlock` argument to
get magicked in, resulting in the following antijoin for the `forall`:

```
[2021-10-04 12:07:46] (108s) Tuple counts for GuardedControlFlow::ConditionBlock::controls_dispred#fbf#antijoin_rhs/5@d84e94 after 1m44s:
201222345 ~7%     {5} r1 = JOIN GuardedControlFlow::ConditionBlock::controls_dispred#fbf#shared#2 WITH Flow::BasicBlock::getASuccessor_dispred#ff_10#join_rhs ON FIRST 1 OUTPUT Lhs.0 'arg1', Rhs.1 'arg4', Lhs.1 'arg0', Lhs.2 'arg2', Lhs.3 'arg3'
200599933 ~4%     {5} r2 = JOIN r1 WITH Flow::BasicBlock::dominates#ff ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.0 'arg1', Lhs.3 'arg2', Lhs.4 'arg3', Lhs.1 'arg4'

0         ~0%     {4} r3 = JOIN GuardedControlFlow::ConditionBlock::controls_dispred#fbf#shared#1 WITH GuardedControlFlow::ConditionBlock#class#f ON FIRST 1 OUTPUT Lhs.0 'arg3', Lhs.2 'arg1', Lhs.1 'arg0', false

0         ~0%     {4} r4 = JOIN GuardedControlFlow::ConditionBlock::controls_dispred#fbf#shared WITH GuardedControlFlow::ConditionBlock#class#f ON FIRST 1 OUTPUT Lhs.0 'arg3', Lhs.2 'arg1', Lhs.1 'arg0', true

0         ~0%     {4} r5 = r3 UNION r4
0         ~0%     {5} r6 = JOIN r5 WITH Flow::BasicBlock::getASuccessor_dispred#ff ON FIRST 2 OUTPUT Lhs.2 'arg0', Lhs.1 'arg1', Lhs.3 'arg2', Lhs.0 'arg3', Rhs.0

200599933 ~4%     {5} r7 = r2 UNION r6
                return r7
```
(cancelled)

I observed that quick-eval'ing the `controls` predicate exhibit no such
bad join order (and terminated quickly) which lead me to conclude that
this was a case of bad magic.

Adding the `pragma[nomagic]` resulted in a return to the previous
performance.
2021-10-04 12:16:53 +00:00
Rasmus Lerchedahl Petersen
aa91c26792 Python: Add missing taint steps 2021-10-04 12:12:07 +02:00
yoff
4521a9fdf0 Update python/ql/lib/semmle/python/frameworks/Stdlib.qll
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-10-04 11:36:53 +02:00
yoff
1ce9426adf Merge pull request #6761 from RasmusWL/cryptodome-sha3
Python/JS: Recognize SHA-3 hash functions
2021-10-01 13:33:36 +02:00
Anders Schack-Mulligen
98f68cb053 Dataflow: Sync. 2021-10-01 13:11:43 +02:00
Anders Schack-Mulligen
490df2027b Dataflow: Add language-specific predicate forceHighPrecision(). 2021-10-01 13:11:14 +02:00
Rasmus Lerchedahl Petersen
175a06fe73 Python: Fix compile error due to predicate rename 2021-10-01 10:33:42 +02:00
Rasmus Lerchedahl Petersen
35d9005eae Python: typo again.. 2021-09-30 14:39:44 +02:00
Rasmus Lerchedahl Petersen
f3fc56a167 Python: typos 2021-09-30 14:39:05 +02:00
Rasmus Lerchedahl Petersen
d19d37bf9b Python: more suggestions from review 2021-09-30 14:36:26 +02:00
yoff
c1c63d0c28 Merge pull request #6738 from RasmusWL/qldoc-getArgByName
Python: Add QLDoc to `Function.getArgByName`
2021-09-30 14:11:18 +02:00
yoff
46e62cd963 Apply suggestions from code review
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com>
2021-09-30 14:00:18 +02:00
Rasmus Lerchedahl Petersen
02e91b3902 Python: Model functions that will raise
on non-existing files.
2021-09-30 13:36:24 +02:00
Rasmus Wriedt Larsen
ba990f72f2 Another hasLocationInfo URL reference fix 2021-09-29 14:00:28 +02:00
Rasmus Wriedt Larsen
987b573709 Fix hasLocationInfo URL reference
Follow up to https://github.com/github/codeql/pull/5830
2021-09-29 13:47:58 +02:00
Rasmus Wriedt Larsen
ded3088529 Python/JS: Recognize SHA-3 hash functions
Official names are SHA3-224, SHA3-256, SHA3-384, SHA3-512 as per
https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
2021-09-27 12:08:40 +02:00
Rasmus Lerchedahl Petersen
f2fbeed490 Python: Model os.path-functions 2021-09-23 15:30:00 +02:00
Rasmus Wriedt Larsen
d4564d5dd1 Python: Add QLDoc to Function.getArgByName 2021-09-23 10:01:04 +02:00
Erik Krogh Kristensen
805d1d170c do not filter away regular expressions with lookbehinds 2021-09-22 17:14:29 +02:00
Rasmus Wriedt Larsen
8badba26b8 Python: Minor SQLALchemy comment fixes 2021-09-22 13:58:29 +02:00
Rasmus Wriedt Larsen
a83bb39d0f Python: Merge SQLAlchemy TextClause injection into py/sql-injection
As discussed in a meeting today, this will end up presenting an query
suite that's easier to use for customers.

Since https://github.com/github/codeql/pull/6589 has JUST been merged,
if we get this change in fast enough, no end-user will ever have run
`py/sqlalchemy-textclause-injection` as part of LGTM.com or Code
Scanning.
2021-09-21 20:21:42 +02:00
Anders Schack-Mulligen
dd1bed02e8 Merge pull request #6721 from aschackmull/dataflow/subpaths01-joinorder
Dataflow: Fix join-order in subpaths01
2021-09-21 16:05:41 +02:00
Rasmus Wriedt Larsen
c7c8e2f3e3 Merge branch 'main' into promote-sqlalchemy 2021-09-21 09:36:07 +02:00
Anders Schack-Mulligen
044623a360 Dataflow: Sync. 2021-09-20 14:58:28 +02:00
Rasmus Lerchedahl Petersen
64685f31dc Python: Add missing qldoc
Also do some general cleanup
How was this allowed comitted in the first place?
2021-09-16 16:51:43 +02:00
Rasmus Lerchedahl Petersen
72bf390ec5 Merge branch 'main' of github.com:github/codeql into python/promote-regex-injection 2021-09-16 14:50:17 +02:00
Tom Hvitved
1c1c46591e Merge pull request #6708 from hvitved/python/files-folders-drop-columns
Python: Drop redundant columns from `files` and `folders` relations
2021-09-16 14:42:15 +02:00
Tom Hvitved
9f10018d48 Address review comment 2021-09-16 13:11:03 +02:00
Taus
783233dfe4 Merge pull request #6696 from yoff/python/copy-multiples-performance-fix-from-ruby
Python: Copy performance fix for `multiples` from ruby
2021-09-16 13:01:07 +02:00
Tom Hvitved
94b5c4eada Python: Drop redundant columns from files and folders relations 2021-09-16 10:51:27 +02:00
Anders Schack-Mulligen
c0fd44c909 Dataflow: Sync. 2021-09-15 16:10:54 +02:00
Rasmus Lerchedahl Petersen
d37c14880f Python: Copy performance fix 2021-09-14 15:15:50 +02:00
Rasmus Wriedt Larsen
8b7fad8595 Merge pull request #6283 from tausbn/python-fix-exceptstmt-gettype
Python: Fix `ExceptStmt::getType`
2021-09-14 13:40:33 +02:00
Rasmus Wriedt Larsen
49f5f1e2c2 Merge pull request #6336 from tausbn/python-make-annotated-assignment-a-definitionnode
Python: Two fixes regarding annotated assignments
2021-09-14 13:37:53 +02:00
Rasmus Lerchedahl Petersen
abbd1d1dc5 Python: Fix errors introduced during port
testing on a database helps..
2021-09-14 13:08:21 +02:00
Rasmus Lerchedahl Petersen
6c82daef3d Python: Move Regexinjection out of experimental
and fix up structure
2021-09-14 11:54:59 +02:00
Rasmus Lerchedahl Petersen
3d5192d6d3 Python: Fix typos 2021-09-14 11:54:11 +02:00
Rasmus Lerchedahl Petersen
a30f697537 Python: Add getName to RegexExecution concept 2021-09-14 11:53:40 +02:00
Taus
b51ce1d2b3 Merge pull request #6640 from yoff/python-add-parameter-default-value-flow-step
Python: add parameter default value flow step
2021-09-13 17:05:48 +02:00
Rasmus Lerchedahl Petersen
8f152a5bfb Python: Port regex concepts and adapt PolyRedos 2021-09-13 16:50:00 +02:00
Rasmus Wriedt Larsen
db78e3a7da Merge pull request #6274 from tausbn/python-api-graphs-import-star
Python: Support `import *` in API graphs
2021-09-10 13:25:41 +02:00
Rasmus Wriedt Larsen
b45743b562 Merge pull request #6312 from tausbn/python-deprecate-importnode
Python: Deprecate `importNode`
2021-09-10 13:12:56 +02:00
Tom Hvitved
649c2ce188 Merge pull request #6586 from hvitved/dataflow/stage2-precise-call-ctx-take2
Data flow: Add precise call contexts to stage 2
2021-09-10 11:34:35 +02:00
Tom Hvitved
296d10fe2a Data flow: Adjust callMayFlowThroughFwd pragmas 2021-09-10 09:21:24 +02:00
Rasmus Lerchedahl Petersen
e27b3162e5 Python: rewrite simpleLocalFlowStep
to take into account the split between
import time and runtime.
2021-09-09 12:43:08 +02:00