yoff
62598c0fd1
Update python/ql/lib/semmle/python/security/dataflow/LogInjectionCustomizations.qll
...
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com >
2022-02-14 16:07:40 +01:00
Rasmus Lerchedahl Petersen
84447e4710
python: more detailed alert message
2022-02-14 11:55:07 +01:00
Rasmus Lerchedahl Petersen
bd14adefa0
python: add apologetic comment
2022-02-14 11:37:46 +01:00
Chuan-kai Lin
9b4dbb9dd8
Merge pull request #7895 from github/cklin/upgrades-initial-dbscheme
...
Upgrade scripts testing: set initial dbschemes
2022-02-11 11:06:12 -08:00
Taus
d7f30de5b0
Merge pull request #7874 from RasmusWL/set-store-step
...
Python: Fix setStoreStep to use `SetElementContent`
2022-02-11 12:50:02 +01:00
github-actions[bot]
21bf29353f
Post-release preparation for codeql-cli-2.8.1
2022-02-11 11:07:31 +00:00
github-actions[bot]
f25fc70b7c
Release preparation for version 2.8.1
2022-02-10 22:08:24 +00:00
Tom Hvitved
58d90c7f8d
Python: More points-to performance improvements
2022-02-10 10:29:30 +01:00
Tom Hvitved
7fd8d6dd30
Address review comments
2022-02-10 10:29:30 +01:00
Tom Hvitved
2de892bfd8
Python: Points-to performance improvements
2022-02-10 10:29:30 +01:00
Rasmus Lerchedahl Petersen
313f9f056c
python: switch to using concepts
2022-02-09 14:36:48 +01:00
Rasmus Lerchedahl Petersen
17aa2898f9
python: model (xpathEval from) libxml2
2022-02-09 14:25:43 +01:00
Rasmus Lerchedahl Petersen
e8649d8947
python: model (etree from) lxml
2022-02-09 14:15:17 +01:00
Tom Hvitved
9440a45015
Merge branch 'main' into post-release-prep/codeql-cli-2.8.0
2022-02-09 09:40:33 +01:00
yoff
f21ac04285
Update python/ql/lib/semmle/python/frameworks/Stdlib.qll
...
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com >
2022-02-09 09:22:31 +01:00
Chuan-kai Lin
a7f1ee574c
Upgrade scripts testing: set initial dbschemes
...
This commit sets initial dbschemes for cpp, csharp, java, javascript, and
python so that automated testing for upgrade scripts would also cover legacy
upgrades.
2022-02-08 11:11:41 -08:00
Rasmus Lerchedahl Petersen
3f36ccba92
python: add name to concept
2022-02-08 12:40:13 +01:00
Rasmus Lerchedahl Petersen
8665fe4817
python: add concept for XPath construction
...
also small fixup in `SqlConstruction`
2022-02-08 12:31:37 +01:00
Rasmus Wriedt Larsen
3e01816f0c
Python: Add change-note
2022-02-08 12:03:40 +01:00
Rasmus Lerchedahl Petersen
7d287f1698
python: add concept for xpath execution
2022-02-08 11:46:28 +01:00
Rasmus Lerchedahl Petersen
a9cfc60ea1
python: move supporting libraries
...
and update reference in query
2022-02-08 11:27:45 +01:00
Rasmus Wriedt Larsen
62702d0ca9
Python: Fix setStoreStep to use SetElementContent
2022-02-07 13:18:36 +01:00
Rasmus Wriedt Larsen
b276b2d48c
Python: Clean up taint steps for attributes
2022-02-07 13:12:31 +01:00
github-actions[bot]
b4ab86c020
Post-release preparation for codeql-cli-2.8.0
2022-02-06 23:34:07 +00:00
yoff
182c62f5c3
Merge pull request #7838 from tausbn/python-fix-charset-performance-problem
...
Python: Fix performance issue in `charSet`
2022-02-04 14:18:13 +01:00
Taus
67be20f368
Python: Remove implied inequalities
...
Also gets rid of `inner_end`, since we're already doing `end - 1 = ...`
in the other fix (and so this is more consistent).
2022-02-04 12:46:06 +00:00
Rasmus Wriedt Larsen
438a01e911
Python: Deprecate old bottle points-to extension
2022-02-04 12:02:09 +01:00
Rasmus Wriedt Larsen
c9e36aaf72
Python: Fix deprecated deprecated
2022-02-04 12:02:09 +01:00
Rasmus Wriedt Larsen
9ec531f040
Python: Add deprecation change-note
2022-02-04 12:02:09 +01:00
Rasmus Wriedt Larsen
84fdd8a739
Python: Add non-deprecated httpVerb to Concepts
2022-02-04 12:02:09 +01:00
Rasmus Wriedt Larsen
5a032d6f84
Python: deprecate old taint-tracking related predicates
2022-02-04 12:02:08 +01:00
Rasmus Wriedt Larsen
dba6b60c80
Python: Deprecate old library modeling
2022-02-04 12:02:08 +01:00
Rasmus Wriedt Larsen
a40fdf7a7c
Python: Deprecate old web modeling
2022-02-04 12:02:08 +01:00
Rasmus Wriedt Larsen
14a1aa0c11
Python: Add change-note
...
I went with `minorAnalysis` instead of `majorAnalysis`, since I don't
think the impact of this change will be major (but that's just my gut
feeling).
2022-02-04 12:00:49 +01:00
Rasmus Wriedt Larsen
b2ce0fcb72
Python: Add post-update nodes to args of unresolved calls
...
Besides solving the problem with `setattr`, it also solved some old
problems with json library modeling (yay).
2022-02-04 11:51:53 +01:00
Rasmus Wriedt Larsen
e9b496ba73
Merge pull request #7831 from RasmusWL/printast-remove-regexp
...
Python: Remove `RegExpTerm` from PrintAST
2022-02-04 11:38:58 +01:00
Erik Krogh Kristensen
5e23da813f
rename named-parameters to keyword-parameters
2022-02-03 23:10:39 +01:00
Erik Krogh Kristensen
e434f075fa
introduce, and use, API::APICallNode
2022-02-03 23:10:39 +01:00
Erik Krogh Kristensen
3801a158a8
remove module exporst nodes from API graphs
2022-02-03 23:10:39 +01:00
Erik Krogh Kristensen
c3f4a851f0
remove some TODOs I won't do
2022-02-03 23:10:39 +01:00
Erik Krogh Kristensen
ef5818e243
support import * in ApiGraphs
2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
16774ba285
add support for named parameters in API graphs
2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
66fd43fc3b
add def edge for function returns
2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
d8eea7ba4c
property writes are def nodes
2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
a908b219e9
more backtracking of def nodes, and lots of tests
2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
038b032a43
get basic module exports to work in API-graphs
2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
df9efbe778
get mimimal def nodes to work in python
2022-02-03 23:10:38 +01:00
Erik Krogh Kristensen
89786d9ce2
rename pr to ref in memberFromRef
2022-02-03 23:10:37 +01:00
Taus
22aa4c9379
Python: Fix performance issue in charSet
...
Observed on `mozilla/bugbug` on the 2.8.0 CLI branch, we had the
following line in the timing report:
```
FullServerSideRequestForgery.ql-17:regex::RegexString::charSet_dispred#fff#antijoin_rhs ............... 1m13s
```
Inspecting the logs, we see the following join:
```
(644s) Tuple counts for regex::RegexString::charSet_dispred#fff#antijoin_rhs/5@f295d1bk after 1m13s:
1 ~0% {1} r1 = CONSTANT(unique string)["]"]
2389 ~4% {3} r2 = JOIN r1 WITH regex::RegexString::nonEscapedCharAt_dispred#fff_201#join_rhs ON FIRST 1 OUTPUT Rhs.1 'arg0', Rhs.2 'arg1', (Rhs.2 'arg1' + 1)
668873 ~0% {6} r3 = JOIN r2 WITH regex::RegexString::char_set_start_dispred#fff ON FIRST 1 OUTPUT Lhs.0 'arg0', "]", Lhs.1 'arg1', Lhs.2 'arg2', Rhs.1 'arg3', Rhs.2 'arg4'
537501371 ~4% {7} r4 = JOIN r3 WITH regex::RegexString::nonEscapedCharAt_dispred#fff_021#join_rhs ON FIRST 2 OUTPUT Lhs.0 'arg0', Lhs.2 'arg1', Lhs.3 'arg2', Lhs.4 'arg3', Lhs.5 'arg4', "]", Rhs.2
269085087 ~0% {7} r5 = SELECT r4 ON In.6 > In.4 'arg4'
89583155 ~3% {7} r6 = SELECT r5 ON In.6 < In.1 'arg1'
89583155 ~26634% {5} r7 = SCAN r6 OUTPUT In.0 'arg0', In.1 'arg1', In.2 'arg2', In.3 'arg3', In.4 'arg4'
return r7
```
Now, this is problematic not just because of the large intermediary join
but also because of the large number of tuples being materialised at the
end. The culprit in this case turns out to be this bit of `charSet`:
```
not exists(int mid | this.nonEscapedCharAt(mid) = "]" | mid > inner_start and mid < inner_end)
```
Rewriting this to instead look for the minimum index at which a `]`
appears resulted in a much nicer join.
I also fixed up a similar issue surrounding the `\N` unicode escape.
Not that I think this will necessarily be relevant, but the `min`-based
solution is more robust either way.
2022-02-03 20:42:04 +00:00
Chuan-kai Lin
c8bc5cfa75
Merge pull request #7825 from github/cklin/python-downgrade-scripts
...
Python: adjust downgrade script location and format
2022-02-03 11:40:07 -08:00
