Initial commit of Python queries and QL libraries.

This commit is contained in:
Mark Shannon
2018-11-19 13:13:39 +00:00
committed by Mark Shannon
parent 90c75cd362
commit 5f58824d1b
725 changed files with 63520 additions and 0 deletions

View File

@@ -0,0 +1,18 @@
class TCPServer(object):
def process_request(self, request, client_address):
self.do_work(request, client_address)
self.shutdown_request(request)
class ThreadingMixIn:
"""Mix-in class to handle each request in a new thread."""
def process_request(self, request, client_address):
"""Start a new thread to process the request."""
t = threading.Thread(target = self.do_work, args = (request, client_address))
t.daemon = self.daemon_threads
t.start()
class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass

View File

@@ -0,0 +1,59 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
When a class subclasses multiple base classes, attribute lookup is performed from left to right amongst the base classes.
This form of attribute lookup is called "method resolution order" and is a solution to the
<a href="http://en.wikipedia.org/wiki/Multiple_inheritance#The_diamond_problem">diamond inheritance problem</a> where several base classes
override a method in a shared superclass.
</p>
<p>
Unfortunately, this means that if more than one base class defines the same attribute, the leftmost base class will effectively override
the attribute of the rightmost base class, even though the leftmost base class is not a subclass of the rightmost base class.
Unless the methods in question are designed for inheritance, using <code>super</code>, then this implicit overriding may not be the desired behavior.
Even if it is the desired behavior it makes the code hard to understand and maintain.
</p>
</overview>
<recommendation>
<p>There are a number of ways that might be used to address this issue:
</p><ul>
<li>Override the attribute in the subclass to implement the correct behavior.</li>
<li>Modify the class hierarchy and move equivalent or redundant methods to a common super class.</li>
<li>Modify the method hierarchy, breaking up complex methods into constituent parts.</li>
<li>Use delegation rather than inheritance.</li>
</ul>
</recommendation>
<example>
<p>
In this example the class <code>ThreadingTCPServer</code> inherits from <code>ThreadingMixIn</code> and from <code>TCPServer</code>.
However, both these classes implement <code>process_request</code> which means that <code>ThreadingTCPServer</code> will inherit
<code>process_request</code> from <code>ThreadingMixIn</code>. Consequently, the implementation of <code>process_request</code> in <code>TCPServer</code>
will be ignored, which may not be the correct behavior.
</p>
<sample src="ConflictingAttributesInBaseClasses.py" />
<p>
This can be fixed either by overriding the method, as shown in class <code>ThreadingTCPServerOverriding</code>
or by ensuring that the
functionality provided by the two base classes does not overlap, as shown in class <code>ThreadingTCPServerChangedHierarchy</code>.
</p>
<sample src="ConflictingAttributesInBaseClasses_Fixed.py" />
</example>
<references>
<li>Python Language Reference: <a href="https://docs.python.org/2/reference/datamodel.html">Data model</a>.</li>
<li>Python releases: <a href="https://www.python.org/download/releases/2.3/mro/">The Python 2.3 Method Resolution Order</a>.</li>
<li>Wikipedia: <a href="https://en.wikipedia.org/wiki/C3_linearization">C3 linearization</a>.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Composition_over_inheritance">Composition over inheritance</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,57 @@
/**
* @name Conflicting attributes in base classes
* @description When a class subclasses multiple base classes and more than one base class defines the same attribute, attribute overriding may result in unexpected behavior by instances of this class.
* @kind problem
* @tags reliability
* maintainability
* modularity
* @problem.severity warning
* @sub-severity low
* @precision high
* @id py/conflicting-attributes
*/
import python
predicate does_nothing(PyFunctionObject f) {
not exists(Stmt s | s.getScope() = f.getFunction() |
not s instanceof Pass and not ((ExprStmt)s).getValue() = f.getFunction().getDocString()
)
}
/* If a method performs a super() call then it is OK as the 'overridden' method will get called */
predicate calls_super(FunctionObject f) {
exists(Call sup, Call meth, Attribute attr, GlobalVariable v |
meth.getScope() = f.getFunction() and
meth.getFunc() = attr and
attr.getObject() = sup and
attr.getName() = f.getName() and
sup.getFunc() = v.getAnAccess() and
v.getId() = "super"
)
}
/** Holds if the given name is white-listed for some reason */
predicate whitelisted(string name) {
/* The standard library specifically recommends this :(
* See https://docs.python.org/3/library/socketserver.html#asynchronous-mixins */
name = "process_request"
}
from ClassObject c, ClassObject b1, ClassObject b2, string name,
int i1, int i2, Object o1, Object o2
where c.getBaseType(i1) = b1 and
c.getBaseType(i2) = b2 and
i1 < i2 and o1 != o2 and
o1 = b1.lookupAttribute(name) and
o2 = b2.lookupAttribute(name) and
not name.matches("\\_\\_%\\_\\_") and
not calls_super(o1) and
not does_nothing(o2) and
not whitelisted(name) and
not o1.overrides(o2) and
not o2.overrides(o1) and
not c.declaresAttribute(name)
select c, "Base classes have conflicting values for attribute '" + name + "': $@ and $@.", o1, o1.toString(), o2, o2.toString()

View File

@@ -0,0 +1,24 @@
#Fixed by overriding. This does not change behavior, but makes it explicit and comprehensible.
class ThreadingTCPServerOverriding(ThreadingMixIn, TCPServer):
def process_request(self, request, client_address):
#process_request forwards to do_work, so it is OK to call ThreadingMixIn.process_request directly
ThreadingMixIn.process_request(self, request, client_address)
#Fixed by separating threading functionality from request handling.
class ThreadingMixIn:
"""Mix-in class to help with threads."""
def do_job_in_thread(self, job, args):
"""Start a new thread to do the job"""
t = threading.Thread(target = job, args = args)
t.start()
class ThreadingTCPServerChangedHierarchy(ThreadingMixIn, TCPServer):
def process_request(self, request, client_address):
"""Start a new thread to process the request."""
self.do_job_in_thread(self.do_work, (request, client_address))

View File

@@ -0,0 +1,40 @@
class Point(object):
def __init__(self, x, y):
self._x = x
self._y = y
def __repr__(self):
return 'Point(%r, %r)' % (self._x, self._y)
def __eq__(self, other):
if not isinstance(other, Point):
return False
return self._x == other._x and self._y == other._y
class ColorPoint(Point):
def __init__(self, x, y, color):
Point.__init__(self, x, y)
self._color = color
def __repr__(self):
return 'ColorPoint(%r, %r)' % (self._x, self._y, self._color)
#ColorPoint(0, 0, Red) == ColorPoint(0, 0, Green) should be False, but is True.
#Fixed version
class ColorPoint(Point):
def __init__(self, x, y, color):
Point.__init__(self, x, y)
self._color = color
def __repr__(self):
return 'ColorPoint(%r, %r)' % (self._x, self._y, self._color)
def __eq__(self, other):
if not isinstance(other, ColorPoint):
return False
return Point.__eq__(self, other) and self._color = other._color

View File

@@ -0,0 +1,37 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>A class that defines attributes that are not present in its superclasses
may need to override the <code>__eq__()</code> method (<code>__ne__()</code>
should also be defined).</p>
<p>Adding additional attributes without overriding <code>__eq__()</code> means
that the additional attributes will not be accounted for in equality tests.</p>
</overview>
<recommendation>
<p>Override the <code>__eq__</code> method.</p>
</recommendation>
<example>
<p>In the following example the <code>ColorPoint</code>
class subclasses the <code>Point</code> class and adds a new attribute,
but does not override the <code>__eq__</code> method.
</p>
<sample src="DefineEqualsWhenAddingAttributes.py" />
</example>
<references>
<li>Peter Grogono, Philip Santas: <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.48.5109&amp;rep=rep1&amp;type=pdf">Equality in Object Oriented Languages</a></li>
</references>
</qhelp>

View File

@@ -0,0 +1,52 @@
/**
* @name __eq__ not overridden when adding attributes
* @description When adding new attributes to instances of a class, equality for that class needs to be defined.
* @kind problem
* @tags reliability
* correctness
* @problem.severity warning
* @sub-severity high
* @precision high
* @id py/missing-equals
*/
import python
import semmle.python.SelfAttribute
import Equality
predicate class_stores_to_attribute(ClassObject cls, SelfAttributeStore store, string name) {
exists(FunctionObject f | f = cls.declaredAttribute(_) and store.getScope() = f.getFunction() and store.getName() = name) and
/* Exclude classes used as metaclasses */
not cls.getASuperType() = theTypeType()
}
predicate should_override_eq(ClassObject cls, Object base_eq) {
not cls.declaresAttribute("__eq__") and
exists(ClassObject sup | sup = cls.getABaseType() and sup.declaredAttribute("__eq__") = base_eq |
not exists(GenericEqMethod eq | eq.getScope() = sup.getPyClass()) and
not exists(IdentityEqMethod eq | eq.getScope() = sup.getPyClass()) and
not base_eq.(FunctionObject).getFunction() instanceof IdentityEqMethod and
not base_eq = theObjectType().declaredAttribute("__eq__")
)
}
/** Does the non-overridden __eq__ method access the attribute,
* which implies that the __eq__ method does not need to be overridden.
*/
predicate superclassEqExpectsAttribute(ClassObject cls, PyFunctionObject base_eq, string attrname) {
not cls.declaresAttribute("__eq__") and
exists(ClassObject sup | sup = cls.getABaseType() and sup.declaredAttribute("__eq__") = base_eq |
exists(SelfAttributeRead store |
store.getName() = attrname |
store.getScope() = base_eq.getFunction()
)
)
}
from ClassObject cls, SelfAttributeStore store, Object base_eq
where class_stores_to_attribute(cls, store, _) and should_override_eq(cls, base_eq) and
/* Don't report overridden unittest.TestCase. -- TestCase overrides __eq__, but subclasses do not really need to. */
not cls.getASuperType().getName() = "TestCase" and
not superclassEqExpectsAttribute(cls, base_eq, store.getName())
select cls, "The class '" + cls.getName() + "' does not override $@, but adds the new attribute $@.", base_eq, "'__eq__'", store, store.getName()

View File

@@ -0,0 +1,71 @@
import python
private Attribute dictAccess(LocalVariable var) {
result.getName() = "__dict__" and
result.getObject() = var.getAnAccess()
}
private Call getattr(LocalVariable obj, LocalVariable attr) {
result.getFunc().(Name).getId() = "getattr" and
result.getArg(0) = obj.getAnAccess() and
result.getArg(1) = attr.getAnAccess()
}
/** A generic equality method that compares all attributes in its dict,
* or compares attributes using `getattr`. */
class GenericEqMethod extends Function {
GenericEqMethod() {
this.getName() = "__eq__" and
exists(LocalVariable self, LocalVariable other |
self.getAnAccess() = this.getArg(0) and self.getId() = "self" and
other.getAnAccess() = this.getArg(1) and
exists(Compare eq |
eq.getOp(0) instanceof Eq or
eq.getOp(0) instanceof NotEq |
// `self.__dict__ == other.__dict__`
eq.getAChildNode() = dictAccess(self) and
eq.getAChildNode() = dictAccess(other)
or
// `getattr(self, var) == getattr(other, var)`
exists(Variable var |
eq.getAChildNode() = getattr(self, var) and
eq.getAChildNode() = getattr(other, var)
)
)
)
}
}
/** An __eq__ method that just does self is other */
class IdentityEqMethod extends Function {
IdentityEqMethod() {
this.getName() = "__eq__" and
exists(LocalVariable self, LocalVariable other |
self.getAnAccess() = this.getArg(0) and self.getId() = "self" and
other.getAnAccess() = this.getArg(1) and
exists(Compare eq | eq.getOp(0) instanceof Is |
eq.getAChildNode() = self.getAnAccess() and
eq.getAChildNode() = other.getAnAccess()
)
)
}
}
/** An (in)equality method that delegates to its complement */
class DelegatingEqualityMethod extends Function {
DelegatingEqualityMethod() {
exists(Return ret, UnaryExpr not_, Compare comp, Cmpop op, Parameter p0, Parameter p1 |
ret.getScope() = this and
ret.getValue() = not_ and
not_.getOp() instanceof Not and not_.getOperand() = comp and
comp.compares(p0.getVariable().getAnAccess(), op, p1.getVariable().getAnAccess()) |
this.getName() = "__eq__" and op instanceof NotEq or
this.getName() = "__ne__" and op instanceof Eq
)
}
}

View File

@@ -0,0 +1,52 @@
# Incorrect: equality method defined but class contains no hash method
class Point(object):
def __init__(self, x, y):
self._x = x
self._y = y
def __repr__(self):
return 'Point(%r, %r)' % (self._x, self._y)
def __eq__(self, other):
if not isinstance(other, Point):
return False
return self._x == other._x and self._y == other._y
# Improved: equality and hash method defined (inequality method still missing)
class PointUpdated(object):
def __init__(self, x, y):
self._x = x
self._y = y
def __repr__(self):
return 'Point(%r, %r)' % (self._x, self._y)
def __eq__(self, other):
if not isinstance(other, Point):
return False
return self._x == other._x and self._y == other._y
def __hash__(self):
return hash(self._x) ^ hash(self._y)
# Improved: equality method defined and class instances made unhashable
class UnhashablePoint(object):
def __init__(self, x, y):
self._x = x
self._y = y
def __repr__(self):
return 'Point(%r, %r)' % (self._x, self._y)
def __eq__(self, other):
if not isinstance(other, Point):
return False
return self._x == other._x and self._y == other._y
#Tell the interpreter that instances of this class cannot be hashed
__hash__ = None

View File

@@ -0,0 +1,46 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>In order to conform to the object model, classes that define their own equality method should also
define their own hash method, or be unhashable. If the hash method is not defined then the <code>hash</code> of the
super class is used. This is unlikely to result in the expected behavior.</p>
<p>A class can be made unhashable by setting its <code>__hash__</code> attribute to <code>None</code>.</p>
<p>In Python 3, if you define a class-level equality method and omit a <code>__hash__</code> method
then the class is automatically marked as unhashable.</p>
</overview>
<recommendation>
<p>When you define an <code>__eq__</code> method for a class, remember to implement a <code>__hash__</code> method or set
<code>__hash__ = None</code>.</p>
</recommendation>
<example>
<p>In the following example the <code>Point</code> class defines an equality method but
no hash method. If hash is called on this class then the hash method defined for <code>object</code>
is used. This is unlikely to give the required behavior. The <code>PointUpdated</code> class
is better as it defines both an equality and a hash method.
If <code>Point</code> was not to be used in <code>dict</code>s or <code>set</code>s, then it could be defined as
<code>UnhashablePoint</code> below.
</p>
<p>
To comply fully with the object model this class should also define an inequality method (identified
by a separate rule).</p>
<sample src="EqualsOrHash.py" />
</example>
<references>
<li>Python Language Reference: <a href="http://docs.python.org/reference/datamodel.html#object.__hash__">object.__hash__</a>.</li>
<li>Python Glossary: <a href="http://docs.python.org/2/glossary.html#term-hashable">hashable</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,46 @@
/**
* @name Inconsistent equality and hashing
* @description Defining equality for a class without also defining hashability (or vice-versa) violates the object model.
* @kind problem
* @tags reliability
* correctness
* external/cwe/cwe-581
* @problem.severity warning
* @sub-severity high
* @precision very-high
* @id py/equals-hash-mismatch
*/
import python
FunctionObject defines_equality(ClassObject c, string name) {
(name = "__eq__" or major_version() = 2 and name = "__cmp__")
and
result = c.declaredAttribute(name)
}
FunctionObject implemented_method(ClassObject c, string name) {
result = defines_equality(c, name)
or
result = c.declaredAttribute("__hash__") and name = "__hash__"
}
string unimplemented_method(ClassObject c) {
not exists(defines_equality(c, _)) and
(result = "__eq__" and major_version() = 3 or major_version() = 2 and result = "__eq__ or __cmp__")
or
/* Python 3 automatically makes classes unhashable if __eq__ is defined, but __hash__ is not */
not c.declaresAttribute(result) and result = "__hash__" and major_version() = 2
}
predicate violates_hash_contract(ClassObject c, string present, string missing, Object method) {
not c.unhashable() and
missing = unimplemented_method(c) and
method = implemented_method(c, present) and
not c.unknowableAttributes()
}
from ClassObject c, string present, string missing, FunctionObject method
where violates_hash_contract(c, present, missing, method) and
exists(c.getPyClass()) // Suppress results that aren't from source
select method, "Class $@ implements " + present + " but does not define " + missing + ".", c, c.getName()

View File

@@ -0,0 +1,32 @@
class PointOriginal(object):
def __init__(self, x, y):
self._x, x
self._y = y
def __repr__(self):
return 'Point(%r, %r)' % (self._x, self._y)
def __eq__(self, other): # Incorrect: equality is defined but inequality is not
if not isinstance(other, Point):
return False
return self._x == other._x and self._y == other._y
class PointUpdated(object):
def __init__(self, x, y):
self._x, x
self._y = y
def __repr__(self):
return 'Point(%r, %r)' % (self._x, self._y)
def __eq__(self, other):
if not isinstance(other, Point):
return False
return self._x == other._x and self._y == other._y
def __ne__(self, other): # Improved: equality and inequality method defined (hash method still missing)
return not self == other

View File

@@ -0,0 +1,37 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>In order to conform to the object model, classes should define either no equality methods, or both
an equality and an inequality method. If only one of <code>__eq__</code> or <code>__ne__</code> is
defined then the method from the super class is used. This is unlikely to result in the expected
behavior.</p>
</overview>
<recommendation>
<p>When you define an equality or an inequality method for a class, remember to implement both an
<code>__eq__</code> method and an <code>__ne__</code> method.</p>
</recommendation>
<example>
<p>In the following example the <code>PointOriginal</code> class defines an equality method but
no inequality method. If this class is tested for inequality then a type error will be raised. The
<code>PointUpdated</code> class is better as it defines both an equality and an inequality method. To
comply fully with the object model this class should also define a hash method (identified by
a separate rule).</p>
<sample src="EqualsOrNotEquals.py" />
</example>
<references>
<li>Python Language Reference: <a href="http://docs.python.org/2/reference/datamodel.html#object.__ne__">object.__ne__</a>,
<a href="http://docs.python.org/2/reference/expressions.html#comparisons">Comparisons</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,50 @@
/**
* @name Inconsistent equality and inequality
* @description Defining only an equality method or an inequality method for a class violates the object model.
* @kind problem
* @tags reliability
* correctness
* @problem.severity warning
* @sub-severity high
* @precision very-high
* @id py/inconsistent-equality
*/
import python
import Equality
string equals_or_ne() {
result = "__eq__" or result = "__ne__"
}
predicate total_ordering(Class cls) {
exists(Attribute a | a = cls.getADecorator() |
a.getName() = "total_ordering")
or
exists(Name n | n = cls.getADecorator() |
n.getId() = "total_ordering")
}
FunctionObject implemented_method(ClassObject c, string name) {
result = c.declaredAttribute(name) and name = equals_or_ne()
}
string unimplemented_method(ClassObject c) {
not c.declaresAttribute(result) and result = equals_or_ne()
}
predicate violates_equality_contract(ClassObject c, string present, string missing, FunctionObject method) {
missing = unimplemented_method(c) and
method = implemented_method(c, present) and
not c.unknowableAttributes() and
not total_ordering(c.getPyClass()) and
/* Python 3 automatically implements __ne__ if __eq__ is defined, but not vice-versa */
not (major_version() = 3 and present = "__eq__" and missing = "__ne__") and
not method.getFunction() instanceof DelegatingEqualityMethod and
not c.lookupAttribute(missing).(FunctionObject).getFunction() instanceof DelegatingEqualityMethod
}
from ClassObject c, string present, string missing, FunctionObject method
where violates_equality_contract(c, present, missing, method)
select method, "Class $@ implements " + present + " but does not implement " + missing + ".", c, c.getName()

View File

@@ -0,0 +1,6 @@
class IncompleteOrdering(object):
def __init__(self, i):
self.i = i
def __lt__(self, other):
return self.i < other.i

View File

@@ -0,0 +1,35 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p> A class that implements an ordering operator
(<code>__lt__</code>, <code>__gt__</code>, <code>__le__</code> or <code>__ge__</code>) should implement
all four in order that ordering between two objects is consistent and obeys the usual mathematical rules.
If the ordering is inconsistent with default equality, then <code>__eq__</code> and <code>__ne__</code>
should also be implemented.
</p>
</overview>
<recommendation>
<p>Ensure that all four ordering comparisons are implemented as well as <code>__eq__</code> and <code>
__ne__</code> if required.</p>
<p>It is not necessary to manually implement all four comparisons,
the <code>functools.total_ordering</code> class decorator can be used.</p>
</recommendation>
<example>
<p>In this example only the <code>__lt__</code> operator has been implemented which could lead to
inconsistent behavior. <code>__gt__</code>, <code>__le__</code>, <code>__ge__</code>, and in this case,
<code>__eq__</code> and <code>__ne__</code> should be implemented.</p>
<sample src="IncompleteOrdering.py" />
</example>
<references>
<li>Python Language Reference: <a href="http://docs.python.org/2/reference/datamodel.html#object.__lt__">Rich comparisons in Python</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,75 @@
/**
* @name Incomplete ordering
* @description Class defines one or more ordering method but does not define all 4 ordering comparison methods
* @kind problem
* @tags reliability
* correctness
* @problem.severity warning
* @sub-severity low
* @precision very-high
* @id py/incomplete-ordering
*/
import python
predicate total_ordering(Class cls) {
exists(Attribute a | a = cls.getADecorator() |
a.getName() = "total_ordering")
or
exists(Name n | n = cls.getADecorator() |
n.getId() = "total_ordering")
}
string ordering_name(int n) {
result = "__lt__" and n = 1 or
result = "__le__" and n = 2 or
result = "__gt__" and n = 3 or
result = "__ge__" and n = 4
}
predicate overrides_ordering_method(ClassObject c, string name) {
name = ordering_name(_) and
(
c.declaresAttribute(name)
or
exists(ClassObject sup |
sup = c.getASuperType() and not sup = theObjectType() |
sup.declaresAttribute(name)
)
)
}
string unimplemented_ordering(ClassObject c, int n) {
not c = theObjectType() and
not overrides_ordering_method(c, result) and
result = ordering_name(n)
}
string unimplemented_ordering_methods(ClassObject c, int n) {
n = 0 and result = "" and exists(unimplemented_ordering(c, _))
or
exists(string prefix, int nm1 |
n = nm1 + 1 and prefix = unimplemented_ordering_methods(c, nm1) |
prefix = "" and result = unimplemented_ordering(c, n)
or
result = prefix and not exists(unimplemented_ordering(c, n)) and n < 5
or
prefix != "" and result = prefix + " or " + unimplemented_ordering(c, n)
)
}
Object ordering_method(ClassObject c, string name) {
/* If class doesn't declare a method then don't blame this class (the superclass will be blamed). */
name = ordering_name(_) and result = c.declaredAttribute(name)
}
from ClassObject c, Object ordering, string name
where not c.unknowableAttributes() and
not total_ordering(c.getPyClass())
and ordering = ordering_method(c, name) and
exists(unimplemented_ordering(c, _))
select c, "Class " + c.getName() + " implements $@, but does not implement " + unimplemented_ordering_methods(c, 4) + ".",
ordering, name

View File

@@ -0,0 +1,6 @@
class X(object):
def __init__(self):
print("X")
class Y(object,X):
def __init__(self):
print("Y")

View File

@@ -0,0 +1,30 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>Python 2.3 introduced new-style classes (classes inheriting from object). New-style classes use
the C3 linearization method to determine a method resolution ordering (MRO) for each class. The C3
linearization method ensures that for a class C, if a class C1 precedes a class C2 in the MRO of C
then C1 should also precede C2 in the MRO of all subclasses of C. It is possible to create a
situation where it is impossible to achieve this consistency and this will guarantee that a <code>
TypeError</code> will be raised at runtime.</p>
</overview>
<recommendation>
<p>Use a class hierarchy that is not ambiguous.</p>
</recommendation>
<example>
<p>The MRO of class <code>X</code> is just <code>X, object</code>. The program will fail when the MRO
of class <code>Y</code> needs to be calculated because <code>object</code> precedes <code>X</code> in
the definition of <code>Y</code> but the opposite is true in the MRO of <code>X</code>.</p>
<sample src="InconsistentMRO.py" />
</example>
<references>
<li>Python: <a href="http://www.python.org/download/releases/2.3/mro/">The Python 2.3 Method Resolution Order</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,27 @@
/**
* @name Inconsistent method resolution order
* @description Class definition will raise a type error at runtime due to inconsistent method resolution order(MRO)
* @kind problem
* @tags reliability
* correctness
* @problem.severity error
* @sub-severity high
* @precision very-high
* @id py/inconsistent-mro
*/
import python
ClassObject left_base(ClassObject type, ClassObject base) {
exists(int i | i > 0 and type.getBaseType(i) = base and result = type.getBaseType(i-1))
}
predicate invalid_mro(ClassObject t, ClassObject left, ClassObject right) {
t.isNewStyle() and
left = left_base(t, right) and left = right.getAnImproperSuperType()
}
from ClassObject t, ClassObject left, ClassObject right
where invalid_mro(t, left, right)
select t, "Construction of class " + t.getName() + " can fail due to invalid method resolution order(MRO) for bases $@ and $@.",
left, left.getName(), right, right.getName()

View File

@@ -0,0 +1,48 @@
#Superclass __init__ calls subclass method
class Super(object):
def __init__(self, arg):
self._state = "Not OK"
self.set_up(arg)
self._state = "OK"
def set_up(self, arg):
"Do some set up"
class Sub(Super):
def __init__(self, arg):
Super.__init__(self, arg)
self.important_state = "OK"
def set_up(self, arg):
Super.set_up(self, arg)
"Do some more set up" # Dangerous as self._state is "Not OK"
#Improved version with inheritance:
class Super(object):
def __init__(self, arg):
self._state = "Not OK"
self.super_set_up(arg)
self._state = "OK"
def super_set_up(self, arg):
"Do some set up"
class Sub(Super):
def __init__(self, arg):
Super.__init__(self, arg)
self.sub_set_up(self, arg)
self.important_state = "OK"
def sub_set_up(self, arg):
"Do some more set up"

View File

@@ -0,0 +1,42 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
When an instance of a class is initialized, the super-class state should be
fully initialized before it becomes visible to the subclass.
Calling methods of the subclass in the superclass' <code>__init__</code>
method violates this important invariant.
</p>
</overview>
<recommendation>
<p>Do not use methods that are subclassed in the construction of an object.
For simpler cases move the initialization into the superclass' <code>__init__</code> method,
preventing it being overridden. Additional initialization of subclass should
be done in the <code>__init__</code> method of the subclass.
For more complex cases, it is advisable to use a static method or function to manage
object creation.
</p>
<p>Alternatively, avoid inheritance altogether using composition instead.</p>
</recommendation>
<example>
<sample src="InitCallsSubclassMethod.py" />
</example>
<references>
<li>CERT Secure Coding: <a href="https://www.securecoding.cert.org/confluence/display/java/MET05-J.+Ensure+that+constructors+do+not+call+overridable+methods">
Rule MET05-J</a>. Although this is a Java rule it applies to most object-oriented languages.</li>
<li>Python Standard Library: <a href="http://docs.python.org/library/functions.html#staticmethod">Static methods</a>.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Composition_over_inheritance">Composition over inheritance</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,35 @@
/**
* @name __init__ method calls overridden method
* @description Calling a method from __init__ that is overridden by a subclass may result in a partially
* initialized instance being observed.
* @kind problem
* @tags reliability
* correctness
* @problem.severity warning
* @sub-severity low
* @precision high
* @id py/init-calls-subclass
*/
import python
from ClassObject supercls, string method, Call call,
FunctionObject overriding, FunctionObject overridden
where
exists(FunctionObject init, SelfAttribute sa |
supercls.declaredAttribute("__init__") = init and
call.getScope() = init.getFunction() and call.getFunc() = sa |
sa.getName() = method and
overridden = supercls.declaredAttribute(method) and
overriding.overrides(overridden)
)
select call, "Call to self.$@ in __init__ method, which is overridden by $@.",
overridden, method,
overriding, overriding.descriptiveString()

View File

@@ -0,0 +1,25 @@
class Spam:
def __init__(self):
self.spam = 'spam, spam, spam'
def set_eggs(eggs):
self.eggs = eggs
def __str__(self):
return '%s and %s' % (self.spam, self.eggs) # Maybe uninitialized attribute 'eggs'
#Fixed version
class Spam:
def __init__(self):
self.spam = 'spam, spam, spam'
self.eggs = None
def set_eggs(eggs):
self.eggs = eggs
def __str__(self):
return '%s and %s' % (self.spam, self.eggs) # OK

View File

@@ -0,0 +1,32 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>A possibly non-existent attribute of <code>self</code> is accessed in a method.
The attribute is set in another method of the class, but may be uninitialized if the
method that uses the attribute is called before the one that sets it.
This may result in an <code>AttributeError</code> at run time.
</p>
</overview>
<recommendation>
<p>Ensure that all attributes are initialized in the <code>__init__</code> method.</p>
</recommendation>
<example>
<sample src="MaybeUndefinedClassAttribute.py" />
</example>
<references>
<li>Python Standard Library: <a href="http://docs.python.org/library/exceptions.html#exceptions.AttributeError">exception AttributeError</a>.
</li>
</references>
</qhelp>

View File

@@ -0,0 +1,40 @@
/**
* @name Maybe undefined class attribute
* @description Accessing an attribute of 'self' that is not initialized in the __init__ method may cause an AttributeError at runtime
* @kind problem
* @tags reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision low
* @id py/maybe-undefined-attribute
*/
import python
import semmle.python.SelfAttribute
predicate guarded_by_other_attribute(SelfAttributeRead a, CheckClass c) {
c.sometimesDefines(a.getName()) and
exists(SelfAttributeRead guard, If i |
i.contains(a) and
c.assignedInInit(guard.getName()) |
i.getTest() = guard
or
i.getTest().contains(guard)
)
}
predicate maybe_undefined_class_attribute(SelfAttributeRead a, CheckClass c) {
c.sometimesDefines(a.getName()) and
not c.alwaysDefines(a.getName()) and
c.interestingUndefined(a) and
not guarded_by_other_attribute(a, c)
}
from Attribute a, ClassObject c, SelfAttributeStore sa
where maybe_undefined_class_attribute(a, c) and
sa.getClass() = c.getPyClass() and sa.getName() = a.getName()
select a, "Attribute '" + a.getName() +
"' is not defined in the class body nor in the __init__() method, but it is defined $@", sa, "here"

View File

@@ -0,0 +1,67 @@
import python
// Helper predicates for multiple call to __init__/__del__ queries.
pragma [noinline]
private predicate multiple_invocation_paths(FunctionInvocation top, FunctionInvocation i1, FunctionInvocation i2, FunctionObject multi) {
i1 != i2 and
i1 = top.getACallee+() and
i2 = top.getACallee+() and
i1.getFunction() = multi and
i2.getFunction() = multi
}
/** Holds if `self.name` calls `multi` by multiple paths, and thus calls it more than once. */
predicate multiple_calls_to_superclass_method(ClassObject self, FunctionObject multi, string name) {
exists(FunctionInvocation top, FunctionInvocation i1, FunctionInvocation i2 |
multiple_invocation_paths(top, i1, i2, multi) and
top.runtime(self.declaredAttribute(name)) and
self.getASuperType().declaredAttribute(name) = multi |
/* Only called twice if called from different functions,
* or if one call-site can reach the other */
i1.getCall().getScope() != i2.getCall().getScope()
or
i1.getCall().strictlyReaches(i2.getCall())
)
}
/** Holds if all attributes called `name` can be inferred to be methods. */
private predicate named_attributes_not_method(ClassObject cls, string name) {
cls.declaresAttribute(name) and not cls.declaredAttribute(name) instanceof FunctionObject
}
/** Holds if `f` actually does something. */
private predicate does_something(FunctionObject f) {
f.isBuiltin() and not f = theObjectType().lookupAttribute("__init__")
or
exists(Stmt s | s = f.getFunction().getAStmt() and not s instanceof Pass)
}
/** Holds if `meth` looks like it should have a call to `name`, but does not */
private predicate missing_call(FunctionObject meth, string name) {
exists(CallNode call, AttrNode attr |
call.getScope() = meth.getFunction() and
call.getFunction() = attr and
attr.getName() = name and
not exists(FunctionObject f | f.getACall() = call)
)
}
/** Holds if `self.name` does not call `missing`, even though it is expected to. */
predicate missing_call_to_superclass_method(ClassObject self, FunctionObject top, FunctionObject missing, string name) {
missing = self.getASuperType().declaredAttribute(name) and
top = self.lookupAttribute(name) and
/* There is no call to missing originating from top */
not top.getACallee*() = missing and
/* Make sure that all named 'methods' are objects that we can understand. */
not exists(ClassObject sup |
sup = self.getAnImproperSuperType() and
named_attributes_not_method(sup, name)
) and
not self.isAbstract()
and
does_something(missing)
and
not missing_call(top, name)
}

View File

@@ -0,0 +1,26 @@
class Vehicle(object):
def __del__(self):
recycle(self.base_parts)
class Car(Vehicle):
def __del__(self):
recycle(self.car_parts)
Vehicle.__del__(self)
#Car.__del__ is missed out.
class SportsCar(Car, Vehicle):
def __del__(self):
recycle(self.sports_car_parts)
Vehicle.__del__(self)
#Fix SportsCar by calling Car.__del__
class FixedSportsCar(Car, Vehicle):
def __del__(self):
recycle(self.sports_car_parts)
Car.__del__(self)

View File

@@ -0,0 +1,50 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>Python, unlike statically typed languages such as Java, allows complete freedom when calling methods during object destruction.
However, standard object-oriented principles apply to Python classes using deep inheritance hierarchies.
Therefore the developer has responsibility for ensuring that objects are properly cleaned up when
there are multiple <code>__del__</code> methods that need to be called.
</p>
<p>
If the <code>__del__</code> method of a superclass is not called during object destruction it is likely that
that resources may be leaked.
</p>
<p>A call to the <code>__del__</code> method of a superclass during object destruction may be omitted:
</p>
<ul>
<li>When a subclass calls the <code>__del__</code> method of the wrong class.</li>
<li>When a call to the <code>__del__</code> method of one its base classes is omitted.</li>
</ul>
</overview>
<recommendation>
<p>Either be careful to explicitly call the <code>__del__</code> of the correct base class, or
use <code>super()</code> throughout the inheritance hierarchy.</p>
<p>Alternatively refactor one or more of the classes to use composition rather than inheritance.</p>
</recommendation>
<example>
<p>In this example, explicit calls to <code>__del__</code> are used, but <code>SportsCar</code> erroneously calls
<code>Vehicle.__del__</code>. This is fixed in <code>FixedSportsCar</code> by calling <code>Car.__del__</code>.
</p>
<sample src="MissingCallToDel.py" />
</example>
<references>
<li>Python Tutorial: <a href="https://docs.python.org/2/tutorial/classes.html">Classes</a>.</li>
<li>Python Standard Library: <a href="https://docs.python.org/2/library/functions.html#super">super</a>.</li>
<li>Artima Developer: <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=236275">Things to Know About Python Super</a>.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Composition_over_inheritance">Composition over inheritance</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,26 @@
/**
* @name Missing call to __del__ during object destruction
* @description An omitted call to a super-class __del__ method may lead to class instances not being cleaned up properly.
* @kind problem
* @tags efficiency
* correctness
* @problem.severity error
* @sub-severity low
* @precision high
* @id py/missing-call-to-delete
*/
import python
import MethodCallOrder
from ClassObject self, FunctionObject missing
where
missing_call_to_superclass_method(self, _, missing, "__del__") and
not missing.neverReturns() and
not self.failedInference() and
not missing.isBuiltin()
select self, "Class " + self.getName() + " may not be cleaned up properly as $@ is not called during deletion.",
missing, missing.descriptiveString()

View File

@@ -0,0 +1,26 @@
class Vehicle(object):
def __init__(self):
self.mobile = True
class Car(Vehicle):
def __init__(self):
Vehicle.__init__(self)
self.car_init()
#Car.__init__ is missed out.
class SportsCar(Car, Vehicle):
def __init__(self):
Vehicle.__init__(self)
self.sports_car_init()
#Fix SportsCar by calling Car.__init__
class FixedSportsCar(Car, Vehicle):
def __init__(self):
Car.__init__(self)
self.sports_car_init()

View File

@@ -0,0 +1,52 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>Python, unlike statically typed languages such as Java, allows complete freedom when calling methods during object initialization.
However, standard object-oriented principles apply to Python classes using deep inheritance hierarchies.
Therefore the developer has responsibility for ensuring that objects are properly initialized when
there are multiple <code>__init__</code> methods that need to be called.
</p>
<p>
If the <code>__init__</code> method of a superclass is not called during object initialization it is likely that
that object will end up in an incorrect state.
</p>
<p>A call to the <code>__init__</code> method of a superclass during object initialization may be omitted:
</p>
<ul>
<li>When a subclass calls the <code>__init__</code> method of the wrong class.</li>
<li>When a call to the <code>__init__</code> method of one its base classes is omitted.</li>
<li>When multiple inheritance is used and a class inherits from several base classes,
and at least one of those does not use <code>super()</code> in its own <code>__init__</code> method.</li>
</ul>
</overview>
<recommendation>
<p>Either be careful to explicitly call the <code>__init__</code> of the correct base class, or
use <code>super()</code> throughout the inheritance hierarchy.</p>
<p>Alternatively refactor one or more of the classes to use composition rather than inheritance.</p>
</recommendation>
<example>
<p>In this example, explicit calls to <code>__init__</code> are used, but <code>SportsCar</code> erroneously calls
<code>Vehicle.__init__</code>. This is fixed in <code>FixedSportsCar</code> by calling <code>Car.__init__</code>.
</p>
<sample src="MissingCallToInit.py" />
</example>
<references>
<li>Python Tutorial: <a href="https://docs.python.org/2/tutorial/classes.html">Classes</a>.</li>
<li>Python Standard Library: <a href="https://docs.python.org/2/library/functions.html#super">super</a>.</li>
<li>Artima Developer: <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=236275">Things to Know About Python Super</a>.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Composition_over_inheritance">Composition over inheritance</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,28 @@
/**
* @name Missing call to __init__ during object initialization
* @description An omitted call to a super-class __init__ method may lead to objects of this class not being fully initialized.
* @kind problem
* @tags reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision high
* @id py/missing-call-to-init
*/
import python
import MethodCallOrder
from ClassObject self, FunctionObject initializer, FunctionObject missing
where
self.lookupAttribute("__init__") = initializer and
missing_call_to_superclass_method(self, initializer, missing, "__init__") and
// If a superclass is incorrect, don't flag this class as well.
not missing_call_to_superclass_method(self.getASuperType(), _, missing, "__init__") and
not missing.neverReturns() and
not self.failedInference() and
not missing.isBuiltin() and
not self.isAbstract()
select self, "Class " + self.getName() + " may not be initialized properly as $@ is not called from its $@.",
missing, missing.descriptiveString(), initializer, "__init__ method"

View File

@@ -0,0 +1,40 @@
#This is prone to strange side effects and race conditions.
class MutatingDescriptor(object):
def __init__(self, func):
self.my_func = func
def __get__(self, obj, obj_type):
#Modified state is visible to all instances of C that might call "show".
self.my_obj = obj
return self
def __call__(self, *args):
return self.my_func(self.my_obj, *args)
def show(obj):
print (obj)
class C(object):
def __init__(self, value):
self.value = value
def __str__(self):
return ("C: " + str(self.value))
show = MutatingDescriptor(show)
c1 = C(1)
c1.show()
c2 = C(2)
c2.show()
c1_show = c1.show
c2.show
c1_show()
#Outputs:
#C: 1
#C: 2
#C: 2

View File

@@ -0,0 +1,42 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>The descriptor protocol allows user programmable attribute access.
The descriptor protocol is what enables class methods, static methods, properties and <code>super()</code>.
</p>
<p>
Descriptor objects are class attributes which control the behavior of instance attributes. Consequently, a single descriptor
is common to all instances of a class and should not be mutated when instance attributes are accessed.
</p>
</overview>
<recommendation>
<p>Do not mutate the descriptor object, rather create a new object that contains the necessary state.</p>
</recommendation>
<example>
<p>In this example the descriptor class <code>MutatingDescriptor</code> stores a reference to <code>obj</code> in an attribute.
</p><sample src="MutatingDescriptor.py" />
<p>In the following example, the descriptor class <code>NonMutatingDescriptor</code> returns a new object every time <code>__get__</code>
is called.
</p><sample src="MutatingDescriptorFixed.py" />
</example>
<references>
<li>Python Language Reference: <a href="https://docs.python.org/reference/datamodel.html#descriptors">Implementing Descriptors.</a></li>
<li>Mark Lutz. <em>Learning Python</em>, Section 30.6: Methods Are Objects: Bound or Unbound. O'Reilly 2013.</li>
<li>A real world example: <a href="https://github.com/numpy/numpy/issues/5247">NumPy Issue 5247.</a></li>
</references>
</qhelp>

View File

@@ -0,0 +1,28 @@
/**
* @name Mutation of descriptor in __get__ or __set__ method.
* @description Descriptor objects can be shared across many instances. Mutating them can cause strange side effects or race conditions.
* @kind problem
* @tags reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision very-high
* @id py/mutable-descriptor
*/
import python
predicate mutates_descriptor(ClassObject cls, SelfAttributeStore s) {
cls.isDescriptorType() and
exists(PyFunctionObject f |
cls.lookupAttribute(_) = f and
not f.getName() = "__init__" and
s.getScope() = f.getFunction()
)
}
from ClassObject cls, SelfAttributeStore s
where
mutates_descriptor(cls, s)
select s, "Mutation of descriptor $@ object may lead to action-at-a-distance effects or race conditions for properties.", cls, cls.getName()

View File

@@ -0,0 +1,37 @@
import types
#Immutable version, which is safe to share.
class NonMutatingDescriptor(object):
def __init__(self, func):
self.my_func = func
def __get__(self, obj, obj_type):
#Return a new object to each access.
return types.MethodType(self.my_func, obj)
def show(obj):
print (obj)
class C(object):
def __init__(self, value):
self.value = value
def __str__(self):
return ("C: " + str(self.value))
show = NonMutatingDescriptor(show)
c1 = C(1)
c1.show()
c2 = C(2)
c2.show()
c1_show = c1.show
c2.show
c1_show()
#Outputs:
#C: 1
#C: 2
#C: 1

View File

@@ -0,0 +1,35 @@
#Attribute set in both superclass and subclass
class C(object):
def __init__(self):
self.var = 0
class D(C):
def __init__(self):
self.var = 1 # self.var will be overwritten
C.__init__(self)
class E(object):
def __init__(self):
self.var = 0 # self.var will be overwritten
class F(E):
def __init__(self):
E.__init__(self)
self.var = 1
#Fixed version -- Pass explicitly as a parameter
class G(object):
def __init__(self, var = 0):
self.var = var
class H(G):
def __init__(self):
G.__init__(self, 1)

View File

@@ -0,0 +1,29 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Subclasses should not set attributes that are set in the superclass.
Doing so may violate invariants in the superclass. </p>
</overview>
<recommendation>
<p>
If you did not intend to override the attribute value set in the superclass,
then rename the subclass attribute.
If you do want to be able to set a new value for the attribute of the superclass,
then convert the superclass attribute to a property.
Otherwise the value should be passed as a parameter to the superclass
<code>__init__</code> method.
</p>
</recommendation>
<example>
<sample src="OverwritingAttributeInSuperClass.py" />
</example>
</qhelp>

View File

@@ -0,0 +1,71 @@
/**
* @name Overwriting attribute in super-class or sub-class
* @description Assignment to self attribute overwrites attribute previously defined in subclass or superclass __init__ method.
* @kind problem
* @tags reliability
* maintainability
* modularity
* @problem.severity warning
* @sub-severity low
* @precision medium
* @id py/overwritten-inherited-attribute
*/
import python
class InitCallStmt extends ExprStmt {
InitCallStmt() {
exists(Call call, Attribute attr | call = this.getValue() and attr = call.getFunc() |
attr.getName() = "__init__")
}
}
predicate overwrites_which(Function subinit, AssignStmt write_attr, string which) {
write_attr.getScope() = subinit and self_write_stmt(write_attr, _) and
exists(Stmt top | top.contains(write_attr) or top = write_attr |
(exists(int i, int j, InitCallStmt call | call.getScope() = subinit | i > j and top = subinit.getStmt(i) and call = subinit.getStmt(j) and which = "superclass")
or
exists(int i, int j, InitCallStmt call | call.getScope() = subinit | i < j and top = subinit.getStmt(i) and call = subinit.getStmt(j) and which = "subclass")
)
)
}
predicate self_write_stmt(Stmt s, string attr) {
exists(Attribute a, Name self | self = a.getObject() and s.contains(a) and self.getId() = "self" and a.getCtx() instanceof Store and a.getName() = attr)
}
predicate both_assign_attribute(Stmt s1, Stmt s2, Function f1, Function f2) {
exists(string name | s1.getScope() = f1 and s2.getScope() = f2 and self_write_stmt(s1, name) and self_write_stmt(s2, name))
}
predicate attribute_overwritten(AssignStmt overwrites, AssignStmt overwritten, string name, string classtype, string classname)
{
exists(FunctionObject superinit, FunctionObject subinit, ClassObject superclass, ClassObject subclass, AssignStmt subattr, AssignStmt superattr |
(classtype = "superclass" and classname = superclass.getName() and overwrites = subattr and overwritten = superattr or
classtype = "subclass" and classname = subclass.getName() and overwrites = superattr and overwritten = subattr)
and
/* OK if overwritten in subclass and is a class attribute */
(not exists(superclass.declaredAttribute(name)) or classtype = "subclass")
and
superclass.declaredAttribute("__init__") = superinit
and
subclass.declaredAttribute("__init__") = subinit
and
superclass = subclass.getASuperType()
and
overwrites_which(subinit.getFunction(), subattr, classtype)
and
both_assign_attribute(subattr, superattr, subinit.getFunction(), superinit.getFunction())
and
self_write_stmt(superattr, name)
)
}
from string classtype, AssignStmt overwrites, AssignStmt overwritten, string name, string classname
where attribute_overwritten(overwrites, overwritten, name, classtype, classname)
select overwrites, "Assignment overwrites attribute " + name + ", which was previously defined in " + classtype + " $@.", overwritten, classname

View File

@@ -0,0 +1,42 @@
class OldStyle:
def __init__(self, x):
self._x = x
# Incorrect: 'OldStyle' is not a new-style class and '@property' is not supported
@property
def x(self):
return self._x
class InheritOldStyle(OldStyle):
def __init__(self, x):
self._x = x
# Incorrect: 'InheritOldStyle' is not a new-style class and '@property' is not supported
@property
def x(self):
return self._x
class NewStyle(object):
def __init__(self, x):
self._x = x
# Correct: 'NewStyle' is a new-style class and '@property' is supported
@property
def x(self):
return self._x
class InheritNewStyle(NewStyle):
def __init__(self, x):
self._x = x
# Correct: 'InheritNewStyle' inherits from a new-style class and '@property' is supported
@property
def x(self):
return self._x

View File

@@ -0,0 +1,43 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>Property descriptors are only supported for the new-style classes that were introduced in Python
2.1. Property descriptors should only be used in new-style classes.</p>
</overview>
<recommendation>
<p>If you want to define properties in a class, then ensure that the class is a new-style class. You can
convert an old-style class to a new-style class by inheriting from <code>object</code>.</p>
</recommendation>
<example>
<p>In the following example all the classes attempt to set a property for <code>x</code>. However, only
the third and fourth classes are new-style classes. Consequently, the <code>x</code>
property is only available for the <code>NewStyle</code> and <code>InheritNewStyle</code> classes.</p>
<p>If you define the <code>OldStyle</code> class as inheriting from a new-style class, then the <code>x
</code> property would be available for both the <code>OldStyle</code> and <code>InheritOldStyle</code> classes.</p>
<sample src="PropertyInOldStyleClass.py" />
</example>
<references>
<li>Python Glossary: <a href="http://docs.python.org/glossary.html#term-new-style-class">New-style class</a>.</li>
<li>Python Language Reference: <a href="http://docs.python.org/2/reference/datamodel.html#newstyle">New-style and classic
classes</a>,
<a href="http://docs.python.org/2/reference/datamodel.html#implementing-descriptors">
Descriptors</a>.</li>
<li>Python Standard Library: <a href="http://docs.python.org/library/functions.html#property">Property</a>.</li>
<li>The History of Python: <a href="http://python-history.blogspot.co.uk/2010/06/inside-story-on-new-style-classes.html">
Inside story on new-style classes</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,17 @@
/**
* @name Property in old-style class
* @description Using property descriptors in old-style classes does not work from Python 2.1 onward.
* @kind problem
* @tags portability
* correctness
* @problem.severity error
* @sub-severity low
* @precision very-high
* @id py/property-in-old-style-class
*/
import python
from PropertyObject prop, ClassObject cls
where cls.declaredAttribute(_) = prop and not cls.failedInference() and not cls.isNewStyle()
select prop, "Property " + prop.getName() + " will not work properly, as class " + cls.getName() + " is an old-style class."

View File

@@ -0,0 +1,33 @@
class remotelock(object): # Resources can be released using __del__
def __init__(self, repo):
self.repo = repo
def release(self):
self.repo.unlock()
self.repo = None
def __del__(self):
if self.repo:
self.release()
class remotelock2(object): # Resources can be released using a with statement
def __init__(self, repo):
self.repo = repo
def __enter__(self):
return self
def release(self):
self.repo.unlock()
self.repo = None
def __del__(self):
if self.repo:
self.release()
def __exit__(self, exct_type, exce_value, traceback):
if self.repo:
self.release()

View File

@@ -0,0 +1,42 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>If a class has a <code>close()</code> or similar method to release resources, then it
should be made a context manager. Using a context manager allows instances of the class to be used
in the <code>with</code> statement, improving code size and readability. This is a simpler and more
reliable method than implementing just a <code>__del__</code> method.
</p>
</overview>
<recommendation>
<p>The context manager requires an <code>__enter__</code> and an <code>__exit__</code> method:</p>
<ul>
<li>
<code>__enter__</code> method acquires the resource or does nothing if the resource
is acquired in the <code>__init__</code> method</li>
<li><code>__exit__</code> method releases the resource, this can just be a simple wrapper around the
<code>close</code> method.</li>
</ul>
</recommendation>
<example>
<p>The following example shows how a class definition that implements <code>__del__</code> can be
updated to use a context manager.</p>
<sample src="ShouldBeContextManager.py" />
</example>
<references>
<li>Effbot: <a href="http://effbot.org/zone/python-with-statement.htm">Python with statement</a>.</li>
<li>Python Standard Library: <a href="http://docs.python.org/library/stdtypes.html#context-manager-types">Context manager
</a>.</li>
<li>Python Language Reference: <a href="http://docs.python.org/2.7/reference/datamodel.html#with-statement-context-managers">
With Statement Context Managers</a>.</li>
<li>Python PEP 343: <a href="http://www.python.org/dev/peps/pep-0343">The "with" Statement</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,19 @@
/**
* @name Class should be a context manager
* @description Making a class a context manager allows instances to be used in a 'with' statement.
* This improves resource handling and code readability.
* @kind problem
* @tags maintainability
* readability
* convention
* @problem.severity recommendation
* @sub-severity high
* @precision medium
* @id py/should-be-context-manager
*/
import python
from ClassObject c
where not c.isC() and not c.isContextManager() and exists(c.declaredAttribute("__del__"))
select c, "Class " + c.getName() + " implements __del__ (presumably to release some resource). Consider making it a context manager."

View File

@@ -0,0 +1,20 @@
class Point:
__slots__ = [ '_x', '_y' ] # Incorrect: 'Point' is an old-style class.
# No slots are created.
# Instances of Point have an attribute dictionary.
def __init__(self, x, y):
self._x = x
self._y = y
class Point2(object):
__slots__ = [ '_x', '_y' ] # Correct: 'Point2' is an new-style class
# Two slots '_x' and '_y' are created.
# Instances of Point2 have no attribute dictionary.
def __init__(self, x, y):
self._x = x
self._y = y

View File

@@ -0,0 +1,37 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>The ability to override the class dictionary using a <code>__slots__</code> declaration
is supported only by new-style classes. When you add a <code>__slots__</code> declaration to an
old-style class it just creates a class attribute called '__slots__'.</p>
</overview>
<recommendation>
<p>If you want to override the dictionary for a class, then ensure that the class is a new-style class.
You can convert an old-style class to a new-style class by inheriting from <code>object</code>.</p>
</recommendation>
<example>
<p>In the following example the <code>KeyedRef</code> class is an old-style class (no inheritance). The
<code>__slots__</code> declaration in this class creates a class attribute called '__slots__', the class
dictionary is unaffected. The <code>KeyedRef2</code> class is a new-style class so the
<code>__slots__</code> declaration causes special compact attributes to be created for each name in
the slots list and saves space by not creating attribute dictionaries.</p>
<sample src="SlotsInOldStyleClass.py" />
</example>
<references>
<li>Python Glossary: <a href="http://docs.python.org/glossary.html#term-new-style-class">New-style class</a>.</li>
<li>Python Language Reference: <a href="http://docs.python.org/2/reference/datamodel.html#newstyle">New-style and classic
classes</a>,
<a href="http://docs.python.org/reference/datamodel.html#__slots__">__slots__</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,18 @@
/**
* @name '__slots__' in old-style class
* @description Overriding the class dictionary by declaring '__slots__' is not supported by old-style
* classes.
* @kind problem
* @problem.severity error
* @tags portability
* correctness
* @sub-severity low
* @precision very-high
* @id py/slots-in-old-style-class
*/
import python
from ClassObject c
where not c.isNewStyle() and c.declaresAttribute("__slots__") and not c.failedInference()
select c, "Using __slots__ in an old style class just creates a class attribute called '__slots__'"

View File

@@ -0,0 +1,17 @@
class Mammal(object):
def __init__(self, milk = 0):
self.milk = milk
class Cow(Mammal):
def __init__(self):
Mammal.__init__(self)
def milk(self):
return "Milk"
#Cow().milk() will raise an error as Cow().milk is the 'milk' attribute
#set in Mammal.__init__, not the 'milk' method defined on Cow.

View File

@@ -0,0 +1,27 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p> Subclass shadowing occurs when an instance attribute of a superclass has the
the same name as a method of a subclass, or vice-versa.
The semantics of Python attribute look-up mean that the instance attribute of
the superclass hides the method in the subclass.
</p>
</overview>
<recommendation>
<p>Rename the method in the subclass or rename the attribute in the superclass.</p>
</recommendation>
<example>
<p>The following code includes an example of subclass shadowing. When you call <code>Cow().milk()</code>
an error is raised because <code>Cow().milk</code> is interpreted as the 'milk' attribute set in
<code>Mammal.__init__</code>, not the 'milk' method defined within <code>Cow</code>. This can be fixed
by changing the name of either the 'milk' attribute or the 'milk' method.</p>
<sample src="SubclassShadowing.py" />
</example>
</qhelp>

View File

@@ -0,0 +1,40 @@
/**
* @name Superclass attribute shadows subclass method
* @description Defining an attribute in a superclass method with a name that matches a subclass
* method, hides the method in the subclass.
* @kind problem
* @problem.severity error
* @tags maintainability
* correctness
* @sub-severity low
* @precision high
* @id py/attribute-shadows-method
*/
/* Determine if a class defines a method that is shadowed by an attribute
defined in a super-class
*/
/* Need to find attributes defined in superclass (only in __init__?) */
import python
predicate shadowed_by_super_class(ClassObject c, ClassObject supercls, Assign assign, FunctionObject f)
{
c.getASuperType() = supercls and c.declaredAttribute(_) = f and
exists(FunctionObject init, Attribute attr |
supercls.declaredAttribute("__init__") = init and
attr = assign.getATarget() and
((Name)attr.getObject()).getId() = "self" and
attr.getName() = f.getName() and
assign.getScope() = ((FunctionExpr)init.getOrigin()).getInnerScope()
) and
/* It's OK if the super class defines the method as well.
* We assume that the original method must have been defined for a reason. */
not supercls.hasAttribute(f.getName())
}
from ClassObject c, ClassObject supercls, Assign assign, FunctionObject shadowed
where shadowed_by_super_class(c, supercls, assign, shadowed)
select shadowed.getOrigin(), "Method " + shadowed.getName() + " is shadowed by $@ in super class '"+ supercls.getName() + "'.", assign, "an attribute"

View File

@@ -0,0 +1,18 @@
class PythonModule(_ModuleIteratorHelper): # '_ModuleIteratorHelper' and 'PythonModule' are old-style classes
# class definitions ....
def walkModules(self, importPackages=False):
if importPackages and self.isPackage():
self.load()
return super(PythonModule, self).walkModules(importPackages=importPackages) # super() will fail
class PythonModule2(_ModuleIteratorHelper): # call to super replaced with direct call to class
# class definitions ....
def walkModules(self, importPackages=False):
if importPackages and self.isPackage():
self.load()
return _ModuleIteratorHelper.__init__(PythonModule, self).walkModules(importPackages=importPackages)

View File

@@ -0,0 +1,38 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>The ability to access inherited methods that have been overridden in a class using <code>super()</code>
is supported only by new-style classes. When you use the <code>super()</code> function in an old-style
class it fails.</p>
</overview>
<recommendation>
<p>If you want to access inherited methods using the <code>super()</code> built-in, then ensure that
the class is a new-style class. You can convert an old-style class to a new-style class by inheriting
from <code>object</code>. Alternatively, you can call the <code>__init__</code> method of the superclass
directly from an old-style class using: <code>BaseClass.__init__(...)</code>.</p>
</recommendation>
<example>
<p>In the following example, <code>PythonModule</code> is an old-style class as it inherits from another
old-style class. If the <code>_ModuleIteratorHelper</code> class cannot be converted into a new-style
class, then the call to <code>super()</code> must be replaced. The <code>PythonModule2</code> class
demonstrates the correct way to call a superclass from an old-style class.</p>
<sample src="SuperInOldStyleClass.py" />
</example>
<references>
<li>Python Glossary: <a href="http://docs.python.org/glossary.html#term-new-style-class">New-style class</a>.</li>
<li>Python Language Reference: <a href="http://docs.python.org/2/reference/datamodel.html#newstyle">New-style and classic
classes</a>.</li>
<li>Python Standard Library: <a href="http://docs.python.org/library/functions.html#super">super</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,22 @@
/**
* @name 'super' in old style class
* @description Using super() to access inherited methods is not supported by old-style classes.
* @kind problem
* @tags portability
* correctness
* @problem.severity error
* @sub-severity low
* @precision very-high
* @id py/super-in-old-style
*/
import python
predicate uses_of_super_in_old_style_class(Call s) {
exists(Function f, ClassObject c | s.getScope() = f and f.getScope() = c.getPyClass() and not c.failedInference() and
not c.isNewStyle() and ((Name)s.getFunc()).getId() = "super")
}
from Call c
where uses_of_super_in_old_style_class(c)
select c, "super() will not work in old-style classes"

View File

@@ -0,0 +1,29 @@
#Calling a method multiple times by using explicit calls when a base inherits from other base
class Vehicle(object):
def __del__(self):
recycle(self.base_parts)
class Car(Vehicle):
def __del__(self):
recycle(self.car_parts)
Vehicle.__del__(self)
class SportsCar(Car, Vehicle):
# Vehicle.__del__ will get called twice
def __del__(self):
recycle(self.sports_car_parts)
Car.__del__(self)
Vehicle.__del__(self)
#Fix SportsCar by only calling Car.__del__
class FixedSportsCar(Car, Vehicle):
def __del__(self):
recycle(self.sports_car_parts)
Car.__del__(self)

View File

@@ -0,0 +1,58 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>Python, unlike statically typed languages such as Java, allows complete freedom when calling methods during object destruction.
However, standard object-oriented principles apply to Python classes using deep inheritance hierarchies.
Therefore the developer has responsibility for ensuring that objects are properly cleaned up when
there are multiple <code>__del__</code> methods that need to be called.
</p>
<p>
Calling a <code>__del__</code> method more than once during object destruction risks resources being released multiple
times. The relevant <code>__del__</code> method may not be designed to be called more than once.
</p>
<p>There are a number of ways that a <code>__del__</code> method may be be called more than once.</p>
<ul>
<li>There may be more than one explicit call to the method in the hierarchy of <code>__del__</code> methods.</li>
<li>A class using multiple inheritance directly calls the <code>__del__</code> methods of its base types.
One or more of those base types uses <code>super()</code> to pass down the inheritance chain.</li>
</ul>
</overview>
<recommendation>
<p>Either be careful not to explicitly call a <code>__del__</code> method more than once, or
use <code>super()</code> throughout the inheritance hierarchy.</p>
<p>Alternatively refactor one or more of the classes to use composition rather than inheritance.</p>
</recommendation>
<example>
<p>In the first example, explicit calls to <code>__del__</code> are used, but <code>SportsCar</code> erroneously calls
both <code>Vehicle.__del__</code> and <code>Car.__del__</code>.
This can be fixed by removing the call to <code>Vehicle.__del__</code>, as shown in <code>FixedSportsCar</code>.
</p>
<sample src="SuperclassDelCalledMultipleTimes.py" />
<p>In the second example, there is a mixture of explicit calls to <code>__del__</code> and calls using <code>super()</code>.
To fix this example, <code>super()</code> should be used throughout.
</p>
<sample src="SuperclassInitCalledMultipleTimes2.py" />
</example>
<references>
<li>Python Tutorial: <a href="https://docs.python.org/2/tutorial/classes.html">Classes</a>.</li>
<li>Python Standard Library: <a href="https://docs.python.org/2/library/functions.html#super">super</a>.</li>
<li>Artima Developer: <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=236275">Things to Know About Python Super</a>.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Composition_over_inheritance">Composition over inheritance</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,27 @@
/**
* @name Multiple calls to __del__ during object destruction
* @description A duplicated call to a super-class __del__ method may lead to class instances not be cleaned up properly.
* @kind problem
* @tags efficiency
* correctness
* @problem.severity warning
* @sub-severity high
* @precision very-high
* @id py/multiple-calls-to-delete
*/
import python
import MethodCallOrder
from ClassObject self, FunctionObject multi
where
multiple_calls_to_superclass_method(self, multi, "__del__") and
not multiple_calls_to_superclass_method(self.getABaseType(), multi, "__del__") and
not exists(FunctionObject better |
multiple_calls_to_superclass_method(self, better, "__del__") and
better.overrides(multi)
) and
not self.failedInference()
select self, "Class " + self.getName() + " may not be cleaned up properly as $@ may be called multiple times during destruction.",
multi, multi.descriptiveString()

View File

@@ -0,0 +1,32 @@
#Calling a method multiple times by using explicit calls when a base uses super()
class Vehicle(object):
def __del__(self):
recycle(self.base_parts)
super(Vehicle, self).__del__()
class Car(Vehicle):
def __del__(self):
recycle(self.car_parts)
super(Car, self).__del__()
class SportsCar(Car, Vehicle):
# Vehicle.__del__ will get called twice
def __del__(self):
recycle(self.sports_car_parts)
Car.__del__(self)
Vehicle.__del__(self)
#Fix SportsCar by using super()
class FixedSportsCar(Car, Vehicle):
def __del__(self):
recycle(self.sports_car_parts)
super(SportsCar, self).__del__()

View File

@@ -0,0 +1,36 @@
#Calling a method multiple times by using explicit calls when a base inherits from other base
class Vehicle(object):
def __init__(self):
self.mobile = True
class Car(Vehicle):
def __init__(self):
Vehicle.__init__(self)
self.car_init()
def car_init(self):
pass
class SportsCar(Car, Vehicle):
# Vehicle.__init__ will get called twice
def __init__(self):
Vehicle.__init__(self)
Car.__init__(self)
self.sports_car_init()
def sports_car_init(self):
pass
#Fix SportsCar by only calling Car.__init__
class FixedSportsCar(Car, Vehicle):
def __init__(self):
Car.__init__(self)
self.sports_car_init()
def sports_car_init(self):
pass

View File

@@ -0,0 +1,58 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>Python, unlike statically typed languages such as Java, allows complete freedom when calling methods during object initialization.
However, standard object-oriented principles apply to Python classes using deep inheritance hierarchies.
Therefore the developer has responsibility for ensuring that objects are properly initialized when
there are multiple <code>__init__</code> methods that need to be called.
</p>
<p>
Calling an <code>__init__</code> method more than once during object initialization risks the object being incorrectly initialized.
It is unlikely that the relevant <code>__init__</code> method is designed to be called more than once.
</p>
<p>There are a number of ways that an <code>__init__</code> method may be be called more than once.</p>
<ul>
<li>There may be more than one explicit call to the method in the hierarchy of <code>__init__</code> methods.</li>
<li>A class using multiple inheritance directly calls the <code>__init__</code> methods of its base types.
One or more of those base types uses <code>super()</code> to pass down the inheritance chain.</li>
</ul>
</overview>
<recommendation>
<p>Either be careful not to explicitly call an <code>__init__</code> method more than once, or
use <code>super()</code> throughout the inheritance hierarchy.</p>
<p>Alternatively refactor one or more of the classes to use composition rather than inheritance.</p>
</recommendation>
<example>
<p>In the first example, explicit calls to <code>__init__</code> are used, but <code>SportsCar</code> erroneously calls
both <code>Vehicle.__init__</code> and <code>Car.__init__</code>.
This can be fixed by removing the call to <code>Vehicle.__init__</code>, as shown in <code>FixedSportsCar</code>.
</p>
<sample src="SuperclassInitCalledMultipleTimes.py" />
<p>In the second example, there is a mixture of explicit calls to <code>__init__</code> and calls using <code>super()</code>.
To fix this example, <code>super()</code> should be used throughout.
</p>
<sample src="SuperclassInitCalledMultipleTimes2.py" />
</example>
<references>
<li>Python Tutorial: <a href="https://docs.python.org/2/tutorial/classes.html">Classes</a>.</li>
<li>Python Standard Library: <a href="https://docs.python.org/2/library/functions.html#super">super</a>.</li>
<li>Artima Developer: <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=236275">Things to Know About Python Super</a>.</li>
<li>Wikipedia: <a href="http://en.wikipedia.org/wiki/Composition_over_inheritance">Composition over inheritance</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,26 @@
/**
* @name Multiple calls to __init__ during object initialization
* @description A duplicated call to a super-class __init__ method may lead to objects of this class not being properly initialized.
* @kind problem
* @tags reliability
* correctness
* @problem.severity warning
* @sub-severity high
* @precision very-high
* @id py/multiple-calls-to-init
*/
import python
import MethodCallOrder
from ClassObject self, FunctionObject multi
where multi != theObjectType().lookupAttribute("__init__") and
multiple_calls_to_superclass_method(self, multi, "__init__") and
not multiple_calls_to_superclass_method(self.getABaseType(), multi, "__init__") and
not exists(FunctionObject better |
multiple_calls_to_superclass_method(self, better, "__init__") and
better.overrides(multi)
) and
not self.failedInference()
select self, "Class " + self.getName() + " may not be initialized properly as $@ may be called multiple times during initialization.",
multi, multi.descriptiveString()

View File

@@ -0,0 +1,38 @@
#Calling a method multiple times by using explicit calls when a base uses super()
class Vehicle(object):
def __init__(self):
super(Vehicle, self).__init__()
self.mobile = True
class Car(Vehicle):
def __init__(self):
super(Car, self).__init__()
self.car_init()
def car_init(self):
pass
class SportsCar(Car, Vehicle):
# Vehicle.__init__ will get called twice
def __init__(self):
Vehicle.__init__(self)
Car.__init__(self)
self.sports_car_init()
def sports_car_init(self):
pass
#Fix SportsCar by using super()
class FixedSportsCar(Car, Vehicle):
def __init__(self):
super(SportsCar, self).__init__()
self.sports_car_init()
def sports_car_init(self):
pass

View File

@@ -0,0 +1,19 @@
class Spam:
def __init__(self):
self.spam = 'spam, spam, spam'
def __str__(self):
return '%s and %s' % (self.spam, self.eggs) # Uninitialized attribute 'eggs'
#Fixed version
class Spam:
def __init__(self):
self.spam = 'spam, spam, spam'
self.eggs = None
def __str__(self):
return '%s and %s' % (self.spam, self.eggs) # OK

View File

@@ -0,0 +1,33 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>A non-existent attribute of <code>self</code> is accessed in a method.
An attribute is treated as non-existent if it is not a class attribute
and it is not set in any method of the class.
This may result in an <code>AttributeError</code> at run time.
</p>
</overview>
<recommendation>
<p>Ensure that all attributes are initialized in the <code>__init__</code> method.</p>
</recommendation>
<example>
<sample src="UndefinedClassAttribute.py" />
</example>
<references>
<li>Python Standard Library: <a href="http://docs.python.org/library/exceptions.html#exceptions.AttributeError">exception AttributeError</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,34 @@
/**
* @name Undefined class attribute
* @description Accessing an attribute of 'self' that is not initialized anywhere in the class in the __init__ method may cause an AttributeError at runtime
* @kind problem
* @tags reliability
* correctness
* @problem.severity error
* @sub-severity low
* @precision low
* @id py/undefined-attribute
*/
import python
import semmle.python.SelfAttribute
predicate undefined_class_attribute(SelfAttributeRead a, CheckClass c, int line, string name) {
name = a.getName() and
not c.sometimesDefines(name) and
c.interestingUndefined(a) and
line = a.getLocation().getStartLine() and
not attribute_assigned_in_method(c.getAMethodCalledFromInit(), name)
}
predicate report_undefined_class_attribute(Attribute a, ClassObject c, string name) {
exists(int line |
undefined_class_attribute(a, c, line, name) and
line = min(int x | undefined_class_attribute(_, c, x, name))
)
}
from Attribute a, ClassObject c, string name
where report_undefined_class_attribute(a, c, name)
select a, "Attribute '" + name + "' is not defined in either the class body or in any method"

View File

@@ -0,0 +1,16 @@
class GCDFinder(object):
def __init__(self, a, b):
self.a = a
self.b = b
def calculate(self):
a = self.a
b = self.b
while a != 0 and b != 0:
if a > b:
a = a % b
else:
b = b % a
if a == 0:
return b
return a

View File

@@ -0,0 +1,35 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p> If a class has only one public method (other than its <code>__init__</code>)
it should be replaced with a function.
</p>
</overview>
<recommendation>
<p> Convert the single public method into a function.
If there is an <code>__init__</code> and it sets attributes on the <code>self</code>
then rename the <code>__init__</code> method and remove the <code>self</code> parameter
Make the public method an inner function and return that.</p>
<p>Delete the class.</p>
</recommendation>
<example>
<p>In this example the class only has a single method. This method does not need to be in its own
class. It should be a method on its own that takes <code>a</code> and <code>b</code> as parameters.
</p>
<sample src="UselessClass.py" />
</example>
<references>
<li>Python: <a href="http://docs.python.org/tutorial/classes.html">Classes</a>.</li>
</references>
</qhelp>

View File

@@ -0,0 +1,83 @@
/**
* @name Useless class
* @description Class only defines one public method (apart from __init__ or __new__) and should be replaced by a function
* @kind problem
* @tags maintainability
* useless-code
* @problem.severity recommendation
* @sub-severity low
* @precision medium
* @id py/useless-class
*/
import python
predicate fewer_than_two_public_methods(Class cls, int methods) {
(methods = 0 or methods = 1) and
methods = count(Function f | f = cls.getAMethod() and not f = cls.getInitMethod())
}
predicate does_not_define_special_method(Class cls) {
not exists(Function f | f = cls.getAMethod() and f.isSpecialMethod())
}
predicate no_inheritance(Class c) {
not exists(ClassObject cls, ClassObject other |
cls.getPyClass() = c and
other != theObjectType() |
other.getABaseType() = cls or
cls.getABaseType() = other
)
and
not exists(Expr base | base = c.getABase() |
not base instanceof Name or ((Name)base).getId() != "object"
)
}
predicate is_decorated(Class c) {
exists(c.getADecorator())
}
predicate is_stateful(Class c) {
exists(Function method, ExprContext ctx |
method.getScope() = c and (ctx instanceof Store or ctx instanceof AugStore) |
exists(Subscript s | s.getScope() = method and s.getCtx() = ctx)
or
exists(Attribute a | a.getScope() = method and a.getCtx() = ctx)
)
or
exists(Function method, Call call, Attribute a, string name |
method.getScope() = c and call.getScope() = method and
call.getFunc() = a and a.getName() = name |
name = "pop" or name = "remove" or name = "discard" or
name = "extend" or name = "append"
)
}
predicate useless_class(Class c, int methods) {
c.isTopLevel()
and
c.isPublic()
and
no_inheritance(c)
and
fewer_than_two_public_methods(c, methods)
and
does_not_define_special_method(c)
and
not c.isProbableMixin()
and
not is_decorated(c)
and
not is_stateful(c)
}
from Class c, int methods, string msg
where useless_class(c, methods) and
(methods = 1 and msg = "Class " + c.getName() + " defines only one public method, which should be replaced by a function."
or
methods = 0 and msg = "Class " + c.getName() + " defines no public methods and could be replaced with a namedtuple or dictionary."
)
select c, msg

View File

@@ -0,0 +1,6 @@
class Point(object):
def __init__(self, x, y):
self.x = x
self.y = y
p = Point(x=1, yy=2) # TypeError: 'yy' is not a valid keyword argument

View File

@@ -0,0 +1,35 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
Using a named argument whose name does not correspond to a parameter of the <code>__init__</code> method of the class being instantiated, will result in a
<code>TypeError</code> at runtime.
</p>
</overview>
<recommendation>
<p>Check for typos in the name of the arguments and fix those.
If the name is clearly different, then this suggests a logical error.
The change required to correct the error will depend on whether the wrong argument has been
specified or whether the wrong class has been specified.
</p>
</recommendation>
<example>
<sample src="WrongNameForArgumentInClassInstantiation.py"/>
</example>
<references>
<li>Python Glossary: <a href="https://docs.python.org/2/glossary.html#term-argument">Arguments</a>.</li>
<li>Python Glossary: <a href="https://docs.python.org/glossary.html#term-parameter">Parameters</a>.</li>
<li>Python Programming FAQ: <a href="https://docs.python.org/2/faq/programming.html#faq-argument-vs-parameter">
What is the difference between arguments and parameters?</a>.</li>
<li>The Python Language Reference: <a href="https://docs.python.org/3/reference/datamodel.html#object.__init__">Data model: object.__init__</a></li>
<li>The Python Tutorial: <a href="https://docs.python.org/3/tutorial/classes.html">Classes</a></li>
</references>
</qhelp>

View File

@@ -0,0 +1,27 @@
/**
* @name Wrong name for an argument in a class instantiation
* @description Using a named argument whose name does not correspond to a
* parameter of the __init__ method of the class being
* instantiated, will result in a TypeError at runtime.
* @kind problem
* @tags reliability
* correctness
* external/cwe/cwe-628
* @problem.severity error
* @sub-severity low
* @precision very-high
* @id py/call/wrong-named-class-argument
*/
import python
import Expressions.CallArgs
from Call call, ClassObject cls, string name, FunctionObject init
where
illegally_named_parameter(call, cls, name)
and init = get_function_or_initializer(cls)
select
call, "Keyword argument '" + name + "' is not a supported parameter name of $@.", init, init.getQualifiedName()

View File

@@ -0,0 +1,7 @@
class Point(object):
def __init__(self, x, y):
self.x = x
self.y = y
p = Point(1) # TypeError: too few arguments
p = Point(1,2,3) # TypeError: too many arguments

View File

@@ -0,0 +1,45 @@
<!DOCTYPE qhelp PUBLIC
"-//Semmle//qhelp//EN"
"qhelp.dtd">
<qhelp>
<overview>
<p>
A call to the <code>__init__</code> method of a class must supply an argument
for each parameter that does not have a default value defined, so:
</p>
<ul>
<li>The minimum number of arguments is the number of parameters without default values.</li>
<li>The maximum number of arguments is the total number of parameters, unless
the class <code>__init__</code> method takes a varargs (starred) parameter in
which case there is no limit.</li>
</ul>
</overview>
<recommendation>
<p>If there are too few arguments then check to see which arguments have been omitted and supply values for those.</p>
<p>If there are too many arguments then check to see if any have been added by mistake and remove those.</p>
<p>
Also check where a comma has been inserted instead of an operator or a dot.
For example, the code is <code>obj,attr</code> when it should be <code>obj.attr</code>.
</p>
<p> If it is not clear which are the missing or surplus arguments, then this suggests a logical error.
The fix will then depend on the nature of the error.
</p>
</recommendation>
<example>
<sample src="WrongNumberArgumentsInClassInstantiation.py"/>
</example>
<references>
<li>Python Glossary: <a href="https://docs.python.org/2/glossary.html#term-argument">Arguments</a>.</li>
<li>Python Glossary: <a href="https://docs.python.org/glossary.html#term-parameter">Parameters</a>.</li>
<li>Python Programming FAQ: <a href="https://docs.python.org/2/faq/programming.html#faq-argument-vs-parameter">
What is the difference between arguments and parameters?</a>.</li>
<li>The Python Language Reference: <a href="https://docs.python.org/3/reference/datamodel.html#object.__init__">Data model: object.__init__</a></li>
<li>The Python Tutorial: <a href="https://docs.python.org/3/tutorial/classes.html">Classes</a></li>
</references>
</qhelp>

View File

@@ -0,0 +1,25 @@
/**
* @name Wrong number of arguments in a class instantiation
* @description Using too many or too few arguments in a call to the __init__
* method of a class will result in a TypeError at runtime.
* @kind problem
* @tags reliability
* correctness
* external/cwe/cwe-685
* @problem.severity error
* @sub-severity low
* @precision very-high
* @id py/call/wrong-number-class-arguments
*/
import python
import Expressions.CallArgs
from Call call, ClassObject cls, string too, string should, int limit, FunctionObject init
where
(
too_many_args(call, cls, limit) and too = "too many arguments" and should = "no more than "
or
too_few_args(call, cls, limit) and too = "too few arguments" and should = "no fewer than "
) and init = get_function_or_initializer(cls)
select call, "Call to $@ with " + too + "; should be " + should + limit.toString() + ".", init, init.getQualifiedName()