Rasmus Wriedt Larsen
faebaee141
Python: Use concept tests for XML Parsing
...
I was loosing my mind from looking through those .expected files
Just going to take it one file at time, to make reviewing easier
2022-03-03 20:36:51 +01:00
Rasmus Wriedt Larsen
4b03f5c724
Python: Rename xml.sax test for consistency
2022-03-03 19:39:32 +01:00
Rasmus Wriedt Larsen
7cda901da2
Python: Add separate query for SimpleXMLRPCServer
...
This was a rough quick-n-dirty query, and should get some qhelp as well at some point.
2022-03-03 19:35:33 +01:00
Rasmus Wriedt Larsen
9406a972cd
Python: Fix vuln detection for xml.minidom with parser arg
2022-03-03 17:52:11 +01:00
Rasmus Wriedt Larsen
5a652480b1
Python: Annotate xml.dom tests
2022-03-03 17:37:25 +01:00
Rasmus Wriedt Larsen
c4d08db62a
Python: Expand XML PoC with minidom/pulldom/expat
2022-03-03 17:30:16 +01:00
Rasmus Wriedt Larsen
3affa6cf3a
Python: Annotate xmltodict tests
2022-03-03 15:08:56 +01:00
Rasmus Wriedt Larsen
61291936bf
Python: Properly model xml.etree
2022-03-03 15:06:55 +01:00
Rasmus Wriedt Larsen
703e3e8a0f
Python: Handle DTD retrieval vuln in lxml
2022-03-03 14:46:48 +01:00
Rasmus Wriedt Larsen
e295399f70
Python: Properly handle huge_tree in lxml
2022-03-03 14:43:37 +01:00
Rasmus Wriedt Larsen
124c03c15c
Python: Expand lxml tests
...
And add annotations, see PoC.py for reference
Some of these needs fixing though
2022-03-03 14:40:45 +01:00
Tom Hvitved
9d6d479fba
Add missing QL doc
2022-03-03 14:17:41 +01:00
Rasmus Wriedt Larsen
3c321dd98d
Python: Model lxml.etree.get_default_parser in own class
2022-03-03 13:49:17 +01:00
Rasmus Wriedt Larsen
52891cb476
Python: Add PoC for XML vulns
2022-03-03 13:48:46 +01:00
Tom Hvitved
ba6ff88d05
Sync files
2022-03-03 12:30:50 +01:00
Rasmus Wriedt Larsen
661d8bf553
Python: Better handling of resolve_entities arg in lxml
2022-03-03 10:05:57 +01:00
Rasmus Wriedt Larsen
515b824b3c
Python: Add lxml positive test
2022-03-03 09:42:19 +01:00
Rasmus Lerchedahl Petersen
6946ae931a
python: missed a spot..
2022-03-02 17:12:48 +01:00
Rasmus Wriedt Larsen
7f7758b83d
Python: rewrite xml sax modeling
2022-03-02 15:22:11 +01:00
Rasmus Wriedt Larsen
6dd776b2de
Python: Only produce one alert per vulnerable XML sink
...
This made it much easier to debug the current alerts on tests at least.
Notice that it's important that we have `strictconcat` and not just
`concat`, since `concat` will also allow flow to sinks that are not
vulnerable to any kind of XML vulnerability :|
2022-03-02 15:22:11 +01:00
Rasmus Wriedt Larsen
16e482bf6f
Python: Improve QLDoc for XML parsing/parsers
2022-03-02 14:25:12 +01:00
Rasmus Wriedt Larsen
aaf55b21c4
Python: Add XMLVulnerabilityKind
...
This gives some freedom in changing the name presented, and not worrying about whether you have made a typo that makes everything break :|
2022-03-02 14:25:12 +01:00
Rasmus Wriedt Larsen
ee23c05489
Python: XML: Expose vuln kind on sink
2022-03-02 14:25:12 +01:00
Rasmus Wriedt Larsen
500e0aced6
Python: Rewrite sax XML tests
...
The tests for type-trackers were not that interesting, since they did
not have XML input in both cases, which is the problem we were trying
hard to solve.
I did keep the test-case of not-user-supplied url alive as well though
👍
I added OK/NOT OK annotations.
Notice that we report all 4 kinds of vulnerabilities on line 93
2022-03-02 14:24:46 +01:00
Rasmus Lerchedahl Petersen
143e9ee954
Merge branch 'main' of github.com:github/codeql into python/promote-xpath-injection
2022-03-02 13:14:08 +01:00
Rasmus Lerchedahl Petersen
ee45e79948
python: Create XML modulein Concepts
...
to prepare for XXE and other XML related modelling
2022-03-02 13:10:23 +01:00
Rasmus Lerchedahl Petersen
80be767a7a
python: implement stdlib xpath support
2022-03-02 12:59:34 +01:00
Rasmus Lerchedahl Petersen
06e0f140c5
python: add tests for stdlib xpath
2022-03-02 12:58:37 +01:00
Taus
8460ab4f31
Merge pull request #7549 from hvitved/python/points-to-perf
2022-03-01 23:05:10 +01:00
Rasmus Wriedt Larsen
518e2aeebf
Merge branch 'main' into jorgectf/python/deserialization
2022-03-01 16:47:13 +01:00
Rasmus Lerchedahl Petersen
f55d7d627e
python: model XPathEvaluator
2022-03-01 14:40:13 +01:00
Rasmus Lerchedahl Petersen
3bb17be389
python: add concept and library tests
2022-03-01 14:39:28 +01:00
Tom Hvitved
92fa0071bd
Update python/ql/lib/semmle/python/pointsto/MRO.qll
...
Co-authored-by: Taus <tausbn@github.com >
2022-03-01 14:16:49 +01:00
Tamás Vajk
94cb5c2be4
Merge pull request #8296 from github/post-release-prep/codeql-cli-2.8.2
...
Post-release preparation for codeql-cli-2.8.2
2022-03-01 11:57:36 +01:00
Rasmus Lerchedahl Petersen
ce3ee65f47
python: remove getTree for now
2022-03-01 10:49:21 +01:00
yoff
853857bd7e
Apply suggestions from code review
...
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com >
2022-03-01 10:26:29 +01:00
github-actions[bot]
980f822983
Post-release preparation for codeql-cli-2.8.2
2022-03-01 09:24:30 +00:00
Arthur Baars
5ce6b847d1
Merge pull request #8166 from aibaars/regex-char-sequence-1
...
Ruby/Python: regex parser: group sequences of 'normal' characters
2022-02-28 17:47:53 +01:00
yoff
d953382df9
Merge pull request #7807 from RasmusWL/dataflow-improvements
...
Python: Dataflow improvements
2022-02-28 16:24:00 +01:00
haby0
be40b54b9f
add test
2022-02-28 20:34:58 +08:00
Rasmus Wriedt Larsen
0e0f159891
Python: Use Python 3 for investigation tests
...
Apparently there are minor differences with `test-6-max-import-depth-2`
where under Python 2 `isfile_no_problem.py` still works as before
2022-02-28 11:33:31 +01:00
Rasmus Wriedt Larsen
01d426dc58
Python: Replace rest of from testlib import *
...
I think we should write our tests in a way that puts points-to in the
best condition to resolve calls. Although this specific change did not
change much, it should help set us up for success in the future 👍
2022-02-28 10:58:44 +01:00
Rasmus Wriedt Larsen
ead0b658d2
Python: Fix fieldflow tests by increasing max-import-depth
2022-02-28 10:58:44 +01:00
Rasmus Wriedt Larsen
a0d1cea490
Python: Add investigation of field-flow problem
...
TL;DR; we used a too low value for `--max-import-depth` :(
2022-02-28 10:58:44 +01:00
haby0
b23e28a1e6
add Server-side Request Forgery sinks
2022-02-28 15:24:02 +08:00
Arthur Baars
0c23f5815f
Add change note
2022-02-25 18:43:43 +01:00
Arthur Baars
5044f89105
Ruby/Python re-introduce normalCharacterSequence
2022-02-25 18:43:43 +01:00
Rasmus Wriedt Larsen
f988e1f0d8
Python: Improve field-flow by removing import *
...
Since that apparently impacts call graph resolution with points-to :O
Also interesting that global flow was only not working for those cases
because of the tricky ifs... still need to 100% figure out how those ifs
are messing up the analysis :|
2022-02-25 16:01:08 +01:00
Rasmus Wriedt Larsen
999af15bd5
Python: Show unresolved calls for field-flow tests
2022-02-25 15:58:07 +01:00
Taus
622b32692b
Python: Prevent magic/inlining in getCase
...
This is a simplified version of
https://github.com/github/codeql/pull/8028
consisting of just the `nomagic` fix.
2022-02-25 14:32:59 +00:00
