hohn/codeql
1
0
Fork 0
mirror of https://github.com/github/codeql.git synced 2026-05-05 13:45:19 +02:00
Code Issues Packages Projects Releases Wiki Activity

Python: Implement call-graph with type-trackers

Browse Source 
This commit is a squash of 80 other commits. While developing, things
changed majorly 2-3 times, and it just wasn't feasible to go back and
write a really nice commit history.

My apologies for this HUGE commit.

Also, later on this is where I solved merge conflicts after flow-summaries
PR was merged.

For your amusement, I've included the original commit messages below.

Python: Add proper argument/parameter positions

Python: Handle normal function calls

Python: Reduce dataflow-consistency warnings

Previously there was a lot of failures for `uniqueEnclosingCallable` and
`argHasPostUpdate`

Removing the override of `getEnclosingCallable` in ParameterNode is
probably the most controversial... although from my point of view it's a
change for the better, since we're able to provide data-flow
ParameterNodes for more of the AST parameter nodes.

Python: Adjust `dataflow/calls` test

Python: Implement `isParameterOf`/`argumentOf`/`OutNode`

This makes the tests under `dataflow/basic` work as well 👍

(initially I had these as separate commits, but it felt like it was too much noise)

Python: Accept fix for `dataflow/consistency`

Python: Changes to `coverage/argumentRoutingTest.ql`

Notice we gain a few new resolved arguments.

We loose out on stuff due to:

1. not handling `*` or `**` in either arguments/parameters (yet)
2. not handling special calls (yet)

Python: Small fix for `TestUtil/RoutingTest.qll`

Since the helper predicates do not depend on this, moved outside class.

Python: Accept changes to `dataflow/coverage/NormalDataflowTest.ql`

Most of this is due to:

- not handling any kinds of methods yet
- not handling `*` or `**`

Python: Small investigation of `test_deep_callgraph`

Python: Accept changes to `coverage/localFlow.ql`

I don't fully understand why the .expected file changed.

Since we still have the desired flow, I'm not going to worry too much
about it.

with this commit, the `dataflow/coverage` tests passes 👍

Python: Minor doc update

Python: Add staticmethod/classmethod to `dataflow/calls`

Python: Handle method calls on class instances

without trying to deal with any class inheritance, or
staticmethod/classmethod at all.

Notice that with this change, we only have a DataFlowCall for the calls
that we can actually resolve. I'm not 100% sure if we need to add a
`UnresolvedCall` subclass of `DataFlowCall` for MaD in the future, but
it should be easy to do.

I'm still unsure about the value of `classesCallGraph`, but have just
accepted the changes.

Python: Handle direct method calls `C.foo(C, arg0)`

Python: Handle `@staticmethod`

Python: Handle class method calls... but the code is shit

WIP todo

Rewrite method calls to be better

also fixed a problem with `self` being an argument to the `x.staticmethod()` call :|

Python: Add subclass tests

Python: Split `class_advanced` test

Python: Rewrite call-graph tests to be inline expectation (1/2)

This adds inline expectations, next commit will remove old annotations
code... but I thought it would be easier to review like this.

Minor fixup

Python: Add simple subclass support

Python: more precise subclass lookup

Still not 100% precise.. but it's better

New ambiguous

Python: Add test for `self.m()` and `cls.m()` calls

Python: Handle `self.m()` and `cls.m()` calls

Python: Add tests for `__init__` and `__new__`

Python: Handle class calls

Python: Fix `self` argument passing for class calls

Now field-flow tests also pass 💪 (although the crosstalk
fieldflow test changes were due to this specific commit)

I also copied much of the setup for pre/post update nodes from Ruby,
specifically having the abstract `PostUpdateNodeImpl` in DataFlowPrivate
seemed like a nice change.

Same for the setup with `TNode` definition having the specification
directly in the body, instead of a `NeedsSyntheticPostUpdateNode` class.

Python: Add new crosstalk test WIP

Maybe needs a bit of refactoring, and to see how it all behaves with points-to

Python: Add `super()` call-graph tests

Python: Refactor MethodCall char-pred

In anticipation of supporting `super(MyClass, self).foo()`, where the
`self` argument doesn't come from an AttrNode, but from the second
argument to super.

Without `pragma[inline]` the optimizer found a terrible join-order --
this won't guarantee a good join-order for the future, but for now it
was just so simple and could let me move on with life.

Python: Add basic `super()` support

I debated a little (with myself) whether I should really do
`superTracker`, but I thought "why not" and just rolled with it. I did
not confirm whether it was actually needed anywhere, that is if anyone
does `ref = super; ref().foo()` -- although I certainly doubt it's very
wide-spread.

Python: InlineCallGraphTest: Allow non-unique callable name in different files

Python: more MRO tests

Python: Add MRO approximation for `super()`

Although it's not 100% accurate, it seems to be on level with the one in
points-to.

Python: Remove some spurious targets for direct calls

removal of TODO from refactoring

remove TODOs class call support

Python: Add contrived subclass call example

Python: Remove more spurious call targets

NOTE: I initially forgot to use
`findFunctionAccordingToMroKnownStartingClass` instead of
`findFunctionAccordingToMro` for __init__ and __new__, and since I did
make that mistake myself, I wanted to add something to the test to
highlight this fact, and make it viewable by PR reviewer... this will be
fixed in the next commit.

Python: Proper fix for spurious __init__ targets

Python: Add call-graph example of class decorator

Python: Support decorated classes in new call-graph

Python: Add call-graph tests for `type(obj).meth()`

Python: support `type(obj).meth()`

Python: Add test for callable defined in function

Python: Add test for callable as argument

Current'y we don't find these with type-tracking, which is super
mysterious. I did check that we have proper flow from the arguments to
the parameters.

Python: Found problem for callable as argument :| MAJOR WIP

WIP commit

IT WORKS AGAIN (but terrible performance)

remove pragma[inline]

remove oops

Fix performance problem

I tried to optimize it even further, but I didn't end up achieving anything :|

Fix call-graph comparison

add comparison version with easy lookup

incomplete missing call-graph tests

unhandled tests

trying to replicate missing call-edge due to missing imports ... but it's hard

also seems to be problems with the inline-expectation-value that I used, seems like it has both missing/unexpected results with same value

Python: Add import-problem test

Python: Add shadowing problem

some cleanup of rewrite fix

a little more cleanup

Add consistency queries to call-graph tests

Python: Add post-update nodes for `self` in implicit `super()` uses

But we do need to discuss whether this is the right approach :O

Fix for field-flow tests

This came from more precise argument passing

Fixed results in type-tracking

Comes from better argument passing with super() and handling of
functions with decorators

fix of inline call graph tests

Fixup call annotation test

Many minor cleanups/fixes

NewNormalCall -> NormalCall

Python: Major restructuring + qldoc writing

Python: Accept changes from pre/post update node .toString changes

Python: Reduce `super` complexity !! WIP !!

Python: Only pass self-reference if in same enclosing-callable

Python: Add call-graph test with nested class

This was inspired by the ImpliesDataflow test that showed missing flow
for q_super, but at least for the call-graph, I'm not able to reproduce
this missing result :|

Python: Restrict `super()` to function defined directly on class

Python: Accept fixes to ImpliesDataflow

Python: Expand field-flow crosstalk tests
This commit is contained in:
Rasmus Wriedt Larsen
2022-05-31 01:41:23 +02:00
parent aa78a43407
commit 9c275c177a
81 changed files with 2494 additions and 447 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
python/ql/test/experimental/library-tests/CallGraph/code/aliased_import.py Normal file
View File

@@ -0,0 +1 @@
from .simple import foo

16
python/ql/test/experimental/library-tests/CallGraph/code/bound_method_arg.py Normal file
View File

@@ -0,0 +1,16 @@
class Foo(object):
def meth(self, arg):
print("Foo.meth", arg)
@classmethod
def cm(cls, arg):
print("Foo.cm", arg)
def call_func(func):
func(42) # $ pt,tt=Foo.meth pt,tt=Foo.cm
foo = Foo()
call_func(foo.meth) # $ pt,tt=call_func
call_func(Foo.cm) # $ pt,tt=call_func

55
python/ql/test/experimental/library-tests/CallGraph/code/callable_as_argument.py Normal file
View File

@@ -0,0 +1,55 @@
# ==============================================================================
# function
# ==============================================================================
def call_func(f):
f() # $ pt,tt=my_func pt,tt=test_func.inside_test_func
def my_func():
print("my_func")
call_func(my_func) # $ pt,tt=call_func
def test_func():
def inside_test_func():
print("inside_test_func")
call_func(inside_test_func) # $ pt,tt=call_func
test_func() # $ pt,tt=test_func
# ==============================================================================
# class
# ==============================================================================
def class_func(cls):
cls.sm() # $ pt,tt=MyClass.sm tt=test_class.InsideTestFunc.sm
cls(42) # $ tt=MyClass.__init__ tt=test_class.InsideTestFunc.__init__
class MyClass(object):
def __init__(self, arg):
print(self, arg)
@staticmethod
def sm():
print("MyClass.staticmethod")
class_func(MyClass) # $ pt,tt=class_func
def test_class():
class InsideTestFunc(object):
def __init__(self, arg):
print(self, arg)
@staticmethod
def sm():
print("InsideTestFunc.staticmethod")
class_func(InsideTestFunc) # $ pt,tt=class_func
test_class() # $ pt,tt=test_class

40
python/ql/test/experimental/library-tests/CallGraph/code/class_advanced.py
View File

@@ -1,40 +0,0 @@
class B(object):
def __init__(self, arg):
print('B.__init__', arg)
self._arg = arg
def __str__(self):
print('B.__str__')
return 'B (arg={})'.format(self.arg)
def __add__(self, other):
print('B.__add__')
if isinstance(other, B):
return B(self.arg + other.arg)
return B(self.arg + other)
@property
def arg(self):
print('B.arg getter')
return self._arg
@arg.setter
def arg(self, value):
print('B.arg setter')
self._arg = value
b1 = B(1)
b2 = B(2)
b3 = b1 + b2
print('value printing:', str(b1))
print('value printing:', str(b2))
print('value printing:', str(b3))
b3.arg = 42
b4 = b3 + 100
# this calls `str(b4)` inside
print('value printing:', b4)

30
python/ql/test/experimental/library-tests/CallGraph/code/class_attr_assign.py Normal file
View File

@@ -0,0 +1,30 @@
def my_func():
print("my_func")
class Foo(object):
def __init__(self, func):
self.indirect_ref = func
self.direct_ref = my_func
def later(self):
self.indirect_ref() # $ pt=my_func MISSING: tt=my_func
self.direct_ref() # $ pt=my_func MISSING: tt=my_func
foo = Foo(my_func) # $ tt=Foo.__init__
foo.later() # $ pt,tt=Foo.later
class DummyObject(object):
def method(self):
print("DummyObject.method")
class Bar(object):
def __init__(self):
self.obj = DummyObject()
def later(self):
self.obj.method() # $ pt=DummyObject.method MISSING: tt=DummyObject.method
bar = Bar(my_func) # $ tt=Bar.__init__
bar.later() # $ pt,tt=Bar.later

66
python/ql/test/experimental/library-tests/CallGraph/code/class_construction.py Normal file
View File

@@ -0,0 +1,66 @@
class X(object):
def __init__(self, arg):
print("X.__init__", arg)
X(42) # $ tt=X.__init__
print()
class Y(X):
def __init__(self, arg):
print("Y.__init__", arg)
super().__init__(-arg) # $ pt,tt=X.__init__
Y(43) # $ tt=Y.__init__
print()
# ---
class WithNew(object):
def __new__(cls, arg):
print("WithNew.__new__", arg)
inst = super().__new__(cls)
assert isinstance(inst, cls)
inst.some_method() # $ MISSING: pt,tt=WithNew.some_method
return inst
def __init__(self, arg):
print("WithNew.__init__", arg)
def some_method(self):
print("WithNew.__init__")
WithNew(44) # $ tt=WithNew.__new__ tt=WithNew.__init__
print()
class ExtraCallToInit(object):
def __new__(cls, arg):
print("ExtraCallToInit.__new__", arg)
inst = super().__new__(cls)
assert isinstance(inst, cls)
# you're not supposed to do this, since it will cause the __init__ method will be run twice.
inst.__init__(1001) # $ MISSING: pt,tt=ExtraCallToInit.__init__
return inst
def __init__(self, arg):
print("ExtraCallToInit.__init__", arg, self)
ExtraCallToInit(1000) # $ tt=ExtraCallToInit.__new__ tt=ExtraCallToInit.__init__
print()
class InitNotCalled(object):
"""as described in https://docs.python.org/3/reference/datamodel.html#object.__new__
__init__ will only be called when the returned object from __new__ is an instance of
the `cls` parameter...
"""
def __new__(cls, arg):
print("InitNotCalled.__new__", arg)
return False
def __init__(self, arg):
print("InitNotCalled.__init__", arg)
InitNotCalled(2000) # $ tt=InitNotCalled.__new__ SPURIOUS: tt=InitNotCalled.__init__
print()

34
python/ql/test/experimental/library-tests/CallGraph/code/class_decorator.py Normal file
View File

@@ -0,0 +1,34 @@
# decorated class
def my_class_decorator(cls):
print("dummy decorator")
return cls
@my_class_decorator # $ pt=my_class_decorator tt=my_class_decorator
class A(object):
def foo(self):
pass
a = A()
a.foo() # $ pt,tt=A.foo
class B(A):
def bar(self):
self.foo() # $ pt,tt=A.foo
# decorated class, unknown decorator
from some_unknown_module import unknown_class_decorator
@unknown_class_decorator
class X(object):
def foo(self):
pass
x = X()
x.foo() # $ pt,tt=X.foo
class Y(X):
def bar(self):
self.foo() # $ pt,tt=X.foo

35
python/ql/test/experimental/library-tests/CallGraph/code/class_more_mro.py Normal file
View File

@@ -0,0 +1,35 @@
class Base(object):
def foo(self):
print("Base.foo")
class A(Base):
def foo(self):
print("A.foo")
super().foo() # $ pt,tt=Base.foo
class ASub(A):
pass
class B(Base):
def foo(self):
print("B.foo")
# NOTE: If this missing result is fixed, please update the QLDoc for
# `getNextClassInMro` as well
super().foo() # $ pt,tt=Base.foo MISSING: pt,tt=A.foo
class BSub(B):
def bar(self):
print("BSub.bar")
super().foo() # $ pt,tt=B.foo SPURIOUS: tt=A.foo
bs = BSub()
bs.foo() # $ pt,tt=B.foo
bs.bar() # $ pt,tt=BSub.bar
print("! Indirect")
class Indirect(BSub, ASub):
pass
Indirect().foo() # $ pt,tt=B.foo SPURIOUS: tt=A.foo
Indirect().bar() # $ pt,tt=BSub.bar

22
python/ql/test/experimental/library-tests/CallGraph/code/class_more_mro2.py Normal file
View File

@@ -0,0 +1,22 @@
class A(object):
def foo(self):
print("A.foo")
class B(A):
pass
b = B()
b.foo() # $ pt,tt=A.foo
class C(A):
def foo(self):
print("C.foo")
class BC(B, C):
def bar(self):
print("BC.bar")
super().foo() # $ pt,tt=C.foo SPURIOUS: tt=A.foo
bc = BC()
bc.foo() # $ pt,tt=C.foo SPURIOUS: tt=A.foo
bc.bar() # $ pt,tt=BC.bar

43
python/ql/test/experimental/library-tests/CallGraph/code/class_properties.py Normal file
View File

@@ -0,0 +1,43 @@
class Prop(object):
def __init__(self, arg):
self._arg = arg
self._arg2 = arg
@property
def arg(self):
print('Prop.arg getter')
return self._arg
@arg.setter
def arg(self, value):
print('Prop.arg setter')
self._arg = value
@arg.deleter
def arg(self):
print('Prop.arg deleter')
# haha, you cannot delete me!
def _arg2_getter(self):
print('Prop.arg2 getter')
return self._arg2
def _arg2_setter(self, value):
print('Prop.arg2 setter')
self._arg2 = value
def _arg2_deleter(self):
print('Prop.arg2 deleter')
# haha, you cannot delete me!
arg2 = property(_arg2_getter, _arg2_setter, _arg2_deleter)
prop = Prop(42) # $ tt=Prop.__init__
prop.arg
prop.arg = 43
del prop.arg
prop.arg2
prop.arg2 = 43
del prop.arg2

29
python/ql/test/experimental/library-tests/CallGraph/code/class_simple.py
View File

@@ -1,29 +0,0 @@
class A(object):
def __init__(self, arg):
print('A.__init__', arg)
self.arg = arg
def some_method(self):
print('A.some_method', self)
@staticmethod
def some_staticmethod():
print('A.some_staticmethod')
@classmethod
def some_classmethod(cls):
print('A.some_classmethod', cls)
# TODO: Figure out how to annotate class instantiation (and add one here).
# Current points-to says it's a call to the class (instead of __init__/__new__/metaclass-something).
# However, current test setup uses "callable" for naming, and expects things to be Function.
a = A(42)
a.some_method() # $ pt=A.some_method
a.some_staticmethod() # $ pt=A.some_staticmethod
a.some_classmethod() # $ pt=A.some_classmethod
A.some_staticmethod() # $ pt=A.some_staticmethod
A.some_classmethod() # $ pt=A.some_classmethod

29
python/ql/test/experimental/library-tests/CallGraph/code/class_special_methods.py Normal file
View File

@@ -0,0 +1,29 @@
class B(object):
def __init__(self, arg):
print('B.__init__', arg)
self.arg = arg
def __str__(self):
print('B.__str__')
return 'B (arg={})'.format(self.arg)
def __add__(self, other):
print('B.__add__')
if isinstance(other, B):
return B(self.arg + other.arg) # $ tt=B.__init__
return B(self.arg + other) # $ tt=B.__init__
b = B(1) # $ tt=B.__init__
print(str(b))
# this calls `str(b)` inside
print(b)
b2 = B(2) # $ tt=B.__init__
# __add__ is called
b + b2
b + 100

178
python/ql/test/experimental/library-tests/CallGraph/code/class_subclass.py Normal file
View File

@@ -0,0 +1,178 @@
class A(object):
def __init__(self, arg):
print('A.__init__', arg)
self.arg = arg
def some_method(self):
print('A.some_method', self)
@staticmethod
def some_staticmethod():
print('A.some_staticmethod')
@classmethod
def some_classmethod(cls):
print('A.some_classmethod', cls)
a = A(42) # $ tt=A.__init__
a.some_method() # $ pt,tt=A.some_method
a.some_staticmethod() # $ pt,tt=A.some_staticmethod
a.some_classmethod() # $ pt,tt=A.some_classmethod
A.some_method(a) # $ pt,tt=A.some_method
A.some_staticmethod() # $ pt,tt=A.some_staticmethod
A.some_classmethod() # $ pt,tt=A.some_classmethod
print("- type()")
type(a).some_method(a) # $ pt,tt=A.some_method
type(a).some_staticmethod() # $ pt,tt=A.some_staticmethod
type(a).some_classmethod() # $ pt,tt=A.some_classmethod
# Subclass test
print("\n! B")
class B(A):
pass
b = B(42) # $ tt=A.__init__
b.some_method() # $ pt,tt=A.some_method
b.some_staticmethod() # $ pt,tt=A.some_staticmethod
b.some_classmethod() # $ pt,tt=A.some_classmethod
B.some_method(b) # $ pt,tt=A.some_method
B.some_staticmethod() # $ pt,tt=A.some_staticmethod
B.some_classmethod() # $ pt,tt=A.some_classmethod
print("- type()")
type(b).some_method(b) # $ pt,tt=A.some_method
type(b).some_staticmethod() # $ pt,tt=A.some_staticmethod
type(b).some_classmethod() # $ pt,tt=A.some_classmethod
# Subclass with method override
print("\n! Subclass with method override")
class C(A):
def some_method(self):
print('C.some_method', self)
c = C(42) # $ tt=A.__init__
c.some_method() # $ pt,tt=C.some_method
class D(object):
def some_method(self):
print('D.some_method', self)
class E(C, D):
pass
e = E(42) # $ tt=A.__init__
e.some_method() # $ pt,tt=C.some_method
class F(D, C):
pass
f = F(42) # $ tt=A.__init__
f.some_method() # $ pt,tt=D.some_method
# ------------------------------------------------------------------------------
# self/cls in methods
# ------------------------------------------------------------------------------
class Base(object):
def foo(self):
print('Base.foo')
def bar(self):
print('Base.bar')
def call_stuff(self):
self.foo() # $ pt,tt=Base.foo pt,tt=Sub.foo pt,tt=Mixin.foo
self.bar() # $ pt,tt=Base.bar
self.sm() # $ pt,tt=Base.sm
self.cm() # $ pt,tt=Base.cm
self.sm2() # $ pt,tt=Base.sm2 pt,tt=Sub.sm2
self.cm2() # $ pt,tt=Base.cm2 pt,tt=Sub.cm2
type(self).sm2() # $ pt,tt=Base.sm2 pt,tt=Sub.sm2
type(self).cm2() # $ pt,tt=Base.cm2 pt,tt=Sub.cm2
@staticmethod
def sm():
print("Base.sm")
@classmethod
def cm(cls):
print("Base.cm")
@staticmethod
def sm2():
print("Base.sm2")
@classmethod
def cm2(cls):
print("Base.cm2")
@classmethod
def call_from_cm(cls):
cls.sm() # $ pt,tt=Base.sm
cls.cm() # $ pt,tt=Base.cm
cls.sm2() # $ pt,tt=Base.sm2 pt,tt=Sub.sm2
cls.cm2() # $ pt,tt=Base.cm2 pt,tt=Sub.cm2
base = Base()
print("! base.call_stuff()")
base.call_stuff() # $ pt,tt=Base.call_stuff
print("! Base.call_from_cm()")
Base.call_from_cm() # $ pt,tt=Base.call_from_cm
class Sub(Base):
def foo(self):
print("Sub.foo")
def foo_on_super(self):
sup = super()
sup.foo() # $ pt,tt=Base.foo
def also_call_stuff(self):
self.sm() # $ pt,tt=Base.sm
self.cm() # $ pt,tt=Base.cm
self.sm2() # $ pt,tt=Sub.sm2
self.cm2() # $ pt,tt=Sub.cm2
@staticmethod
def sm2():
print("Sub.sm2")
@classmethod
def cm2(cls):
print("Sub.cm2")
sub = Sub()
print("! sub.foo_on_super()")
sub.foo_on_super() # $ pt,tt=Sub.foo_on_super
print("! sub.call_stuff()")
sub.call_stuff() # $ pt,tt=Base.call_stuff
print("! sub.also_call_stuff()")
sub.also_call_stuff() # $ pt,tt=Sub.also_call_stuff
print("! Sub.call_from_cm()")
Sub.call_from_cm() # $ pt,tt=Base.call_from_cm
class Mixin(object):
def foo(self):
print("Mixin.foo")
class SubWithMixin(Mixin, Base):
# the ordering here means that in Base.call_stuff, the call to self.foo will go to Mixin.foo
pass
swm = SubWithMixin()
print("! swm.call_stuff()")
swm.call_stuff() # $ pt,tt=Base.call_stuff

38
python/ql/test/experimental/library-tests/CallGraph/code/class_subclass2.py Normal file
View File

@@ -0,0 +1,38 @@
class Base(object):
def foo(self):
print("Base.foo")
def call_stuff(self):
print("Base.call_stuff")
self.foo() # $ pt,tt=Base.foo pt,tt=X.foo
class X(object):
def __init__(self):
print("X.__init__")
def foo(self):
print("X.foo")
class Y(object):
def __init__(self):
print("Y.__init__")
def foo(self):
print("Y.foo")
class Contrived(X, Y, Base):
pass
contrived = Contrived() # $ tt=X.__init__
contrived.foo() # $ pt,tt=X.foo
contrived.call_stuff() # $ pt,tt=Base.call_stuff
# Ensure that we don't mix up __init__ resolution for Contrived() due to MRO
# approximation
class HasInit(object):
def __init__(self):
pass
class TryingToTrickYou(Contrived, HasInit):
pass

108
python/ql/test/experimental/library-tests/CallGraph/code/class_super.py Normal file
View File

@@ -0,0 +1,108 @@
def outside_def(self):
print("outside_def")
try:
super().foo()
except RuntimeError:
pass
class A(object):
def foo(self):
print("A.foo")
@classmethod
def bar(cls):
print("A.bar")
class B(A):
def foo(self):
print("B.foo")
def foo_on_super(self):
print("B.foo_on_super")
super().foo() # $ pt,tt=A.foo
super(B, self).foo() # $ pt,tt=A.foo
od = outside_def
@staticmethod
def sm():
try:
super().foo()
except RuntimeError:
print("B.sm")
pass
@classmethod
def bar(cls):
print("B.bar")
@classmethod
def bar_on_super(cls):
print("B.bar_on_super")
super().bar() # $ tt=A.bar
super(B, cls).bar() # $ tt=A.bar
b = B()
b.foo() # $ pt,tt=B.foo
b.foo_on_super() # $ pt,tt=B.foo_on_super
b.od() # $ pt=outside_def
b.sm() # $ pt,tt=B.sm
print("="*10, "static method")
B.bar() # $ pt,tt=B.bar
B.bar_on_super() # $ pt,tt=B.bar_on_super
print("="*10, "Manual calls to super")
super(B, b).foo() # $ pt,tt=A.foo
assert A.foo == super(B, B).foo
super(B, B).foo(b) # $ tt=A.foo
try:
super(B, 42).foo()
except TypeError:
pass
# For some reason, points-to isn't able to resolve any calls from here on. I've tried to
# comment out both try-except blocks, but that did not solve the problem :|
print("="*10, "C")
class C(B):
def foo_on_A(self):
print('C.foo_on_A')
super(B, self).foo() # $ tt=A.foo
c = C()
c.foo_on_A() # $ tt=C.foo_on_A
print("="*10, "Diamon hierachy")
class X(object):
def foo(self):
print('X.foo')
class Y(X):
def foo(self):
print('Y.foo')
super().foo() # $ tt=X.foo
class Z(X):
def foo(self):
print('Z.foo')
super().foo() # $ tt=X.foo tt=Y.foo
print("! z.foo()")
z = Z()
z.foo() # $ tt=Z.foo
class ZY(Z, Y):
pass
print("! zy.foo()")
zy = ZY()
zy.foo() # $ tt=Z.foo

36
python/ql/test/experimental/library-tests/CallGraph/code/conditional_in_argument.py Normal file
View File

@@ -0,0 +1,36 @@
class X(object):
def foo(self, *args):
print("X.foo", args)
def bar(self, *args):
print("X.bar", args)
def func(cond=True):
x = X()
# ok
x.foo() # $ pt,tt=X.foo
x.bar() # $ pt,tt=X.bar
# the conditional in the argument makes us stop tracking the class instance :|
x.foo(1 if cond else 0) # $ pt,tt=X.foo
x.bar() # $ pt=X.bar MISSING: tt=X.bar
func() # $ pt,tt=func
def func2(cond=True):
y = X()
# ok
y.foo() # $ pt,tt=X.foo
y.bar() # $ pt,tt=X.bar
if cond:
arg = 1
else:
arg = 0
y.foo(arg) # $ pt,tt=X.foo
y.bar() # $ pt,tt=X.bar

24
python/ql/test/experimental/library-tests/CallGraph/code/def_in_function.py Normal file
View File

@@ -0,0 +1,24 @@
def test():
def foo():
print("foo")
foo() # $ pt,tt=test.foo
def bar():
print("bar")
def baz():
print("baz")
baz() # $ pt,tt=test.bar.baz
return baz
baz_ref = bar() # $ pt,tt=test.bar
baz_ref() # $ pt,tt=test.bar.baz
class A(object):
def foo(self):
print("A.foo")
a = A()
a.foo() # $ tt=test.A.foo
test() # $ pt,tt=test

43
python/ql/test/experimental/library-tests/CallGraph/code/func_defined_outside_class.py Normal file
View File

@@ -0,0 +1,43 @@
def outside(self):
print("outside", self)
def outside_sm():
print("outside_sm")
def outside_cm(cls):
print("outside_cm", cls)
class A(object):
def foo(self):
print("A.foo")
foo_ref = foo
outside_ref = outside
outside_sm = staticmethod(outside_sm)
outside_cm = classmethod(outside_cm)
a = A()
a.foo_ref() # $ pt=A.foo
a.outside_ref() # $ pt=outside
a.outside_sm() # $ pt=outside_sm
a.outside_cm() # $ pt=outside_cm
# ===
print("\n! B")
# this pattern was seen in django
class B(object):
def _gen(value):
def func(self):
print("B._gen.func", value)
return func
foo = _gen("foo") # $ pt=B._gen
bar = _gen("bar") # $ pt=B._gen
b = B()
b.foo() # $ pt=B._gen.func
b.bar() # $ pt=B._gen.func

87
python/ql/test/experimental/library-tests/CallGraph/code/nested_class.py Normal file
View File

@@ -0,0 +1,87 @@
class A(object):
class B(object):
@staticmethod
def foo():
print("A.B.foo")
@staticmethod
def bar():
print("A.B.bar")
A.B.foo() # $ pt,tt=A.B.foo
A.B.bar() # $ pt,tt=A.B.bar
ab = A.B()
ab.bar() # $ pt,tt=A.B.bar
# ==============================================================================
class OuterBase(object):
def foo(self):
print("OuterBase.foo")
class InnerBase(object):
def foo(self):
print("InnerBase.foo")
class Outer(OuterBase):
def foo(self):
print("Outer.foo")
super().foo() # $ pt,tt=OuterBase.foo
class Inner(InnerBase):
def foo(self):
print("Inner.foo")
super().foo() # $ pt,tt=InnerBase.foo
outer = Outer()
outer.foo() # $ pt,tt=Outer.foo
inner = Outer.Inner()
inner.foo() # $ pt,tt=Outer.Inner.foo
# ==============================================================================
class Base(object):
def foo(self):
print("Base.foo")
class Base2(object):
def foo(self):
print("Base2.foo")
class X(Base):
def meth(self):
print("X.meth")
super().foo() # $ pt,tt=Base.foo
def inner_func():
print("inner_func")
try:
super().foo()
except RuntimeError:
print("RuntimeError, as expected")
inner_func() # $ pt,tt=X.meth.inner_func
def inner_func2(this_works):
print("inner_func2")
super().foo() # $ MISSING: tt=Base.foo
inner_func2(self) # $ pt,tt=X.meth.inner_func2
def class_def_in_func(self):
print("X.class_def_in_func")
class Y(Base2):
def meth(self):
print("Y.meth")
super().foo() # $ pt,tt=Base2.foo
y = Y()
y.meth() # $ tt=X.class_def_in_func.Y.meth
x = X()
x.meth() # $ pt,tt=X.meth
x.class_def_in_func() # $ pt=X.class_def_in_func tt=X.class_def_in_func

7
python/ql/test/experimental/library-tests/CallGraph/code/relative_import.py Normal file
View File

@@ -0,0 +1,7 @@
def test_relative_import():
from .simple import foo
foo() # $ pt,tt="code/simple.py:foo"
def test_aliased_relative_import():
from .aliased_import import foo
foo() # $ pt,tt="code/simple.py:foo"

4
python/ql/test/experimental/library-tests/CallGraph/code/runtime_decision.py
View File

@@ -15,7 +15,7 @@ if len(sys.argv) >= 2 and not sys.argv[1] in ['0', 'False', 'false']:
else:
func = rd_bar
func() # $ pt=rd_foo pt=rd_bar
func() # $ pt,tt=rd_foo pt,tt=rd_bar
# Random doesn't work with points-to :O
if random.random() < 0.5:
@@ -23,4 +23,4 @@ if random.random() < 0.5:
else:
func2 = rd_bar
func2() # $ pt=rd_foo pt=rd_bar
func2() # $ pt,tt=rd_foo pt,tt=rd_bar

22
python/ql/test/experimental/library-tests/CallGraph/code/shadowing.py Normal file
View File

@@ -0,0 +1,22 @@
def foo(n=0):
print("foo", n)
if n > 0:
foo(n-1) # $ pt,tt=foo
foo(1) # $ pt,tt=foo
def test():
def foo():
print("test.foo")
foo() # $ pt,tt=test.foo
class A(object):
def foo(self):
print("A.foo")
foo() # $ pt=foo MISSING: tt=foo
a = A()
a.foo() # $ pt,tt=A.foo

8
python/ql/test/experimental/library-tests/CallGraph/code/simple.py
View File

@@ -12,9 +12,9 @@ def bar():
lam = lambda: print("lambda called")
foo() # $ pt=foo
indirect_foo() # $ pt=foo
bar() # $ pt=bar
lam() # $ pt=lambda[simple.py:12:7]
foo() # $ pt,tt=foo
indirect_foo() # $ pt,tt=foo
bar() # $ pt,tt=bar
lam() # $ pt,tt=lambda[simple.py:12:7]
# python -m trace --trackcalls simple.py

6
python/ql/test/experimental/library-tests/CallGraph/code/through_content.py Normal file
View File

@@ -0,0 +1,6 @@
def my_func():
print("my_func")
funcs = [my_func]
for f in funcs:
f() # $ MISSING: tt=my_func

8
python/ql/test/experimental/library-tests/CallGraph/code/type_tracking_limitation.py Normal file
View File

@@ -0,0 +1,8 @@
def return_arg(arg):
return arg
def my_func():
print("my_func")
x = return_arg(my_func) # $ pt,tt=return_arg
x() # $ pt=my_func MISSING: tt=my_func

8
python/ql/test/experimental/library-tests/CallGraph/code/underscore_prefix_func_name.py
View File

@@ -11,14 +11,14 @@ def some_function():
def _ignored():
print('_ignored')
some_function()
some_function() # $ tt=some_function
def _works_since_called():
print('_works_since_called')
some_function() # $ pt=some_function
some_function() # $ pt,tt=some_function
def works_even_though_not_called():
some_function() # $ pt=some_function
some_function() # $ pt,tt=some_function
globals()['_ignored']()
_works_since_called() # $ pt=_works_since_called
_works_since_called() # $ pt,tt=_works_since_called