Jean Helie
ef49cc1c30
ATM: add XSSThroughDOM boosted query
2022-11-18 16:23:14 -08:00
tiferet
d8ef142ed9
Remove code duplication in query .ql files:
...
Define the query for finding ATM alerts in the base class `AtmConfig`, and call it from each query's .ql file.
2022-11-18 16:07:38 -08:00
tiferet
7476528244
Move the definition of isSink to the base class:
...
Holds if `sink` is a known taint sink or an "effective" sink.
2022-11-18 16:07:38 -08:00
tiferet
818bd27b43
Move the definition of isSource to the base class:
...
A long as we're not boosting sources, `isSource` is identical to `isKnownSource`.
2022-11-18 16:07:36 -08:00
tiferet
fb137c868a
AtmConfig inherits from TaintTracking::Configuration.
...
That way the specific configs which inherit from `AtmConfig` also inherit from `TaintTracking::Configuration`.
This removes the need for two separate config classes for each query.
2022-11-18 16:07:36 -08:00
tiferet
9a37061e45
Oops -- forgot to stage one file in the previous commit :)
2022-11-18 16:06:15 -08:00
tiferet
4a5835d588
Delete StandardEndpointFilters.
...
All remaining functionality in `StandardEndpointFilters` is only being used in `EndpointCharacteristics`, so it can be moved there as a small set of helper predicates.
2022-11-18 16:06:15 -08:00
tiferet
99c47fa626
Delete CoreKnowledge.
...
All remaining functionality in `CoreKnowledge` is only being used in `EndpointCharacteristics`, so it can be moved there as a small set of helper predicates.
2022-11-18 16:06:15 -08:00
tiferet
f88bae36d6
FilteringReason is no longer being used and can be deleted
2022-11-18 16:06:15 -08:00
tiferet
ed59fc39f6
Remove redundant code
...
`isOtherModeledArgument` and `isArgumentToBuiltinFunction` contained the old logic for selecting negative endpoints for training.
These can now be deleted, and replaced by a single base class that collects all EndpointCharacteristics that are currently used to indicate negative training samples: `OtherModeledArgumentCharacteristic`.
This in turn lets us delete code from `StandardEndpointFilters` that effectively said that endpoints that are high-confidence non-sinks shouldn't be scored at inference time, either.
2022-11-18 16:06:14 -08:00
tiferet
8d22fd25f1
Suggestions from code review
2022-11-18 15:57:46 -08:00
tiferet
4a1382925e
Remove some imports that are no longer used
2022-11-16 14:01:16 -08:00
tiferet
ccbf1ca2a9
Add a comment
2022-11-16 13:05:06 -08:00
tiferet
38c40a7192
isEffectiveSink can't be final because ExtractMisclassifiedEndpointFeatures overrides it.
2022-11-16 12:12:50 -08:00
tiferet
8fee9cb0d5
Fix CodeQL warnings
2022-11-16 12:06:52 -08:00
tiferet
c2035e85d2
Be explicit in requiring that each ATM config set its endpoint type.
2022-11-16 11:55:23 -08:00
tiferet
0fd013f9fd
Update the reason names in FilteredTruePositives.expected.
...
This is needed because we changed the names of three endpoint filters that were all called "not a direct argument to a likely external library call or a heuristic sink" in order to disambiguate them (fc56c5a022
2022-11-16 11:54:10 -08:00
tiferet
eab270eb84
Move the definitions of isEffectiveSink and getAReasonSinkExcluded to the base class.
...
They can now be implemented generically for all sink types.
2022-11-16 11:47:24 -08:00
tiferet
fc56c5a022
Implement the type-specific endpoint filters as EndpointCharacteristics.
...
Also disambiguate three filters from three different sink types that all have the same name, "not a direct argument to a likely external library call or a heuristic sink".
2022-11-16 11:14:25 -08:00
tiferet
13cb0ab554
Fix CodeQL warning
2022-11-15 17:32:30 -08:00
tiferet
2ecdfd1ff6
Delete some code that's no longer in use
2022-11-15 17:29:03 -08:00
tiferet
fedb98ddb5
Implement the standard getAReasonSinkExcluded using StandardEndpointFilterCharacteristics
2022-11-15 17:22:00 -08:00
tiferet
cf4e37a0ab
Implement the standard endpoint filters as EndpointCharacteristics
2022-11-15 17:20:20 -08:00
tiferet
cb632b3534
Delete the file ExtractEndpointData.expected which was leftover in the last PR
2022-11-15 17:11:34 -08:00
Tiferet Gazit
710b215c38
Merge pull request #11263 from github/tiferet/extract-training-data
...
ATM: Extract training data
2022-11-15 12:08:13 -08:00
tiferet
fc078a47fd
Apply suggestion from code review
2022-11-15 11:14:01 -08:00
Tiferet Gazit
092e019de9
Apply suggestions from code review
...
Co-authored-by: Stephan Brandauer <kaeluka@github.com >
2022-11-15 10:48:32 -08:00
Andrew Eisenberg
88750a7000
Add more information about ATM queries for external users
2022-11-15 10:17:56 -08:00
Asger F
dc440aaee6
Merge pull request #11255 from asgerf/js/dynamic-import-type-expr
...
JS: Handle DynamicImport in the context of a type
2022-11-15 13:31:08 +01:00
Stephan Brandauer
ec3578364e
remove superfluous class in EndpointCharacteristics hierarchy
2022-11-15 10:17:38 +01:00
tiferet
9ecff0723c
Fix non-ascii character in docs
2022-11-14 16:34:24 -08:00
tiferet
6b7612fed7
Fix import errors in DebugResultInclusion.ql
2022-11-14 15:33:46 -08:00
tiferet
b47723d607
Delete ExtractEndpointData.
...
Also remove the associated test files.
2022-11-14 14:57:59 -08:00
tiferet
9d7e7735d5
Extract training data:
...
Implement the new query that selects data for training. For now we include clauses that implement logic that is identical to the old queries.
Include a temporary wrapper query that converts the resulting data into the format expected by the endpoint pipeline.
Move the small pieces of `ExtractEndpointData` that are still needed into `ExtractEndpointDataTraining.qll`.
2022-11-14 14:33:08 -08:00
Tiferet Gazit
855eddab80
Merge pull request #11174 from github/tiferet/non-sink-endpoint-characteristics
...
Non-sink endpoint characteristics
2022-11-14 09:37:25 -08:00
Asger F
5f18484fa9
JS: Change note
2022-11-14 15:09:30 +01:00
Henry Mercer
afbd05d41a
Merge branch 'main' into codeql-ci/atm/release-0.4.1
2022-11-11 13:19:35 +00:00
github-actions[bot]
b5b69e9357
JS: Bump version of ML-powered library and query packs to 0.4.2
2022-11-11 12:48:00 +00:00
github-actions[bot]
3e5e695325
JS: Bump patch version of ML-powered library and query packs
2022-11-11 12:36:19 +00:00
Erik Krogh Kristensen
90382c4d1c
Merge pull request #11178 from erik-krogh/passcode
...
JS/RB/PY: Recognize `passcode` as sensitive
2022-11-10 17:58:34 +01:00
Erik Krogh Kristensen
724a31b746
fix comment that wasn't updated in test
...
Co-authored-by: Rasmus Wriedt Larsen <rasmuswriedtlarsen@gmail.com >
2022-11-10 15:56:44 +01:00
Michael Nebel
9c6875ec0f
Merge pull request #10777 from michaelnebel/csharp/generatedataextensions
...
C#: Generate data extension files
2022-11-10 13:08:31 +01:00
tiferet
dbcdc2209e
Use names constants for confidence levels
2022-11-09 14:25:08 -08:00
tiferet
b6532fa9a0
Fix QLDoc style warning
2022-11-09 13:10:54 -08:00
tiferet
243980ef73
Documentation improvements
2022-11-09 13:04:16 -08:00
Tiferet Gazit
6cb01a210f
Apply suggestions from code review
...
Co-authored-by: Stephan Brandauer <kaeluka@github.com >
2022-11-09 12:53:52 -08:00
Asger F
83291f378b
Merge pull request #11157 from asgerf/js/yaml-locations
...
JS: fix issue with zero-column yaml locations
2022-11-09 15:57:54 +01:00
Erik Krogh Kristensen
c537c80ed6
Merge pull request #11095 from erik-krogh/exportRead
...
JS: recognize more re-exported values as exported
2022-11-09 12:39:41 +01:00
erik-krogh
23add8a72b
recognize passcode as sensitive
2022-11-09 11:30:57 +01:00
erik-krogh
e0bcfe2afb
add failing test
2022-11-09 11:30:31 +01:00
