python-cheatsheet Documentation, Study notes of Programming Languages

Download python-cheatsheet Documentation and more Study notes Programming Languages in PDF only on Docsity! python-cheatsheet Documentation Release 0.1.0 crazyguitar May 01, 2022 python-cheatsheet Documentation, Release 0.1.0 Welcome to pysheeet. This project aims at collecting useful Python snippets in order to enhance pythoneers’ coding ex- periences. Please feel free to contribute if you have any awesome ideas for improvements to code snippets, explanations, etc. Any snippets are welcome. If you’d like to contribute, fork pysheeet on GitHub. If there is any question or suggestion, please create an issue on GitHub Issues. CONTENTS 1 python-cheatsheet Documentation, Release 0.1.0 2 CONTENTS CHAPTER ONE WHAT’S NEW IN PYTHON 3 The official document, What’s New In Python, displays all of the most important changes. However, if you’re too busy to read the whole changes, this part provides a brief glance of new features in Python 3. 1.1 New in Python3 Table of Contents • New in Python3 – print is a function – String is unicode – Division Operator – New dict implementation – Keyword-Only Arguments – New Super – Remove <> – BDFL retirement – Not allow from module import * inside function – Add nonlocal keyword – Extended iterable unpacking – General unpacking – Function annotations – Variable annotations – Core support for typing module and generic types – Format byte string – fstring – Suppressing exception – Generator delegation – async and await syntax 3 python-cheatsheet Documentation, Release 0.1.0 >>> 1 / 2 0.5 >>> 1 // 2 0 1.1.4 New dict implementation New in Python 3.6 • PEP 468 - Preserving the order of **kwargs in a function • PEP 520 - Preserving Class Attribute Definition Order • bpo 27350 - More compact dictionaries with faster iteration Before Python 3.5 >>> import sys >>> sys.getsizeof({str(i):i for i in range(1000)}) 49248 >>> d = {'timmy': 'red', 'barry': 'green', 'guido': 'blue'} >>> d # without order-preserving {'barry': 'green', 'timmy': 'red', 'guido': 'blue'} Python 3.6 • Memory usage is smaller than Python 3.5 • Preserve insertion ordered >>> import sys >>> sys.getsizeof({str(i):i for i in range(1000)}) 36968 >>> d = {'timmy': 'red', 'barry': 'green', 'guido': 'blue'} >>> d # preserve insertion ordered {'timmy': 'red', 'barry': 'green', 'guido': 'blue'} 1.1.5 Keyword-Only Arguments New in Python 3.0 • PEP 3102 - Keyword-Only Arguments >>> def f(a, b, *, kw): ... print(a, b, kw) ... >>> f(1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: f() takes 2 positional arguments but 3 were given >>> f(1, 2) Traceback (most recent call last): (continues on next page) 6 Chapter 1. What’s New In Python 3 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) File "<stdin>", line 1, in <module> TypeError: f() missing 1 required keyword-only argument: 'kw' >>> f(1, 2, kw=3) 1 2 3 1.1.6 New Super New in Python 3.0 • PEP 3135 - New Super Python 2 >>> class ParentCls(object): ... def foo(self): ... print "call parent" ... >>> class ChildCls(ParentCls): ... def foo(self): ... super(ChildCls, self).foo() ... print "call child" ... >>> p = ParentCls() >>> c = ChildCls() >>> p.foo() call parent >>> c.foo() call parent call child Python 3 >>> class ParentCls(object): ... def foo(self): ... print("call parent") ... >>> class ChildCls(ParentCls): ... def foo(self): ... super().foo() ... print("call child") ... >>> p = ParentCls() >>> c = ChildCls() >>> p.foo() call parent >>> c.foo() call parent call child 1.1. New in Python3 7 python-cheatsheet Documentation, Release 0.1.0 1.1.7 Remove <> New in Python 3.0 Python 2 >>> a = "Python2" >>> a <> "Python3" True # equal to != >>> a != "Python3" True Python 3 >>> a = "Python3" >>> a != "Python2" True 1.1.8 BDFL retirement New in Python 3.1 • PEP 401 - BDFL Retirement >>> from __future__ import barry_as_FLUFL >>> 1 != 2 File "<stdin>", line 1 1 != 2 ^ SyntaxError: with Barry as BDFL, use '<>' instead of '!=' >>> 1 <> 2 True 1.1.9 Not allow from module import * inside function New in Python 3.0 >>> def f(): ... from os import * ... File "<stdin>", line 1 SyntaxError: import * only allowed at module level 8 Chapter 1. What’s New In Python 3 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) >>> o.x [1, 2, 3] >>> o.y {'foo': 'bar'} 1.1.15 Core support for typing module and generic types New in Python 3.7 • PEP 560 - Core support for typing module and generic types Before Python 3.7 >>> from typing import Generic, TypeVar >>> from typing import Iterable >>> T = TypeVar('T') >>> class C(Generic[T]): ... ... >>> def func(l: Iterable[C[int]]) -> None: ... for i in l: ... print(i) ... >>> func([1,2,3]) 1 2 3 Python 3.7 or above >>> from typing import Iterable >>> class C: ... def __class_getitem__(cls, item): ... return f"{cls.__name__}[{item.__name__}]" ... >>> def func(l: Iterable[C[int]]) -> None: ... for i in l: ... print(i) ... >>> func([1,2,3]) 1 2 3 1.1. New in Python3 11 python-cheatsheet Documentation, Release 0.1.0 1.1.16 Format byte string New in Python 3.5 • PEP 461 - Adding % formatting to bytes and bytearray >>> b'abc %b %b' % (b'foo', b'bar') b'abc foo bar' >>> b'%d %f' % (1, 3.14) b'1 3.140000' >>> class Cls(object): ... def __repr__(self): ... return "repr" ... def __str__(self): ... return "str" ... 'repr' >>> b'%a' % Cls() b'repr' 1.1.17 fstring New in Python 3.6 • PEP 498 - Literal String Interpolation >>> py = "Python3" >>> f'Awesome {py}' 'Awesome Python3' >>> x = [1, 2, 3, 4, 5] >>> f'{x}' '[1, 2, 3, 4, 5]' >>> def foo(x:int) -> int: ... return x + 1 ... >>> f'{foo(0)}' '1' >>> f'{123.567:1.3}' '1.24e+02' 1.1.18 Suppressing exception New in Python 3.3 • PEP 409 - Suppressing exception context Without raise Exception from None >>> def func(): ... try: ... 1 / 0 ... except ZeroDivisionError: ... raise ArithmeticError (continues on next page) 12 Chapter 1. What’s New In Python 3 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) ... >>> func() Traceback (most recent call last): File "<stdin>", line 3, in func ZeroDivisionError: division by zero During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in func ArithmeticError With raise Exception from None >>> def func(): ... try: ... 1 / 0 ... except ZeroDivisionError: ... raise ArithmeticError from None ... >>> func() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 5, in func ArithmeticError # debug >>> try: ... func() ... except ArithmeticError as e: ... print(e.__context__) ... division by zero 1.1.19 Generator delegation New in Python 3.3 • PEP 380 - Syntax for Delegating to a Subgenerator >>> def fib(n: int): ... a, b = 0, 1 ... for _ in range(n): ... yield a ... b, a = a + b, b ... >>> def delegate(n: int): ... yield from fib(n) ... (continues on next page) 1.1. New in Python3 13 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) # async for in list >>> async def coro(n: int): ... return [f async for f in fib(n)] ... >>> loop.run_until_complete(coro(5)) [0, 1, 1, 2, 3] # await in list >>> async def slowfmt(n: int) -> str: ... await asyncio.sleep(0.5) ... return f'{n}' ... >>> async def coro(n: int): ... return [await slowfmt(f) async for f in fib(n)] ... >>> loop.run_until_complete(coro(5)) ['0', '1', '1', '2', '3'] 1.1.23 Matrix multiplication New in Python 3.5 • PEP 465 - A dedicated infix operator for matrix multiplication >>> # "@" represent matrix multiplication >>> class Arr: ... def __init__(self, *arg): ... self._arr = arg ... def __matmul__(self, other): ... if not isinstance(other, Arr): ... raise TypeError ... if len(self) != len(other): ... raise ValueError ... return sum([x*y for x, y in zip(self._arr, other._arr)]) ... def __imatmul__(self, other): ... if not isinstance(other, Arr): ... raise TypeError ... if len(self) != len(other): ... raise ValueError ... res = sum([x*y for x, y in zip(self._arr, other._arr)]) ... self._arr = [res] ... return self ... def __len__(self): ... return len(self._arr) ... def __str__(self): ... return self.__repr__() ... def __repr__(self): ... return "Arr({})".format(repr(self._arr)) ... (continues on next page) 16 Chapter 1. What’s New In Python 3 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) >>> a = Arr(9, 5, 2, 7) >>> b = Arr(5, 5, 6, 6) >>> a @ b # __matmul__ 124 >>> a @= b # __imatmul__ >>> a Arr([124]) 1.1.24 Data Classes New in Python 3.7 PEP 557 - Data Classes Mutable Data Class >>> from dataclasses import dataclass >>> @dataclass ... class DCls(object): ... x: str ... y: str ... >>> d = DCls("foo", "bar") >>> d DCls(x='foo', y='bar') >>> d = DCls(x="foo", y="baz") >>> d DCls(x='foo', y='baz') >>> d.z = "bar" Immutable Data Class >>> from dataclasses import dataclass >>> from dataclasses import FrozenInstanceError >>> @dataclass(frozen=True) ... class DCls(object): ... x: str ... y: str ... >>> try: ... d.x = "baz" ... except FrozenInstanceError as e: ... print(e) ... cannot assign to field 'x' >>> try: ... d.z = "baz" ... except FrozenInstanceError as e: ... print(e) ... cannot assign to field 'z' 1.1. New in Python3 17 python-cheatsheet Documentation, Release 0.1.0 1.1.25 Built-in breakpoint() New in Python 3.7 • PEP 553 - Built-in breakpoint() >>> for x in range(3): ... print(x) ... breakpoint() ... 0 > <stdin>(1)<module>()->None (Pdb) c 1 > <stdin>(1)<module>()->None (Pdb) c 2 > <stdin>(1)<module>()->None (Pdb) c 1.1.26 The walrus operator New in Python 3.8 • PEP 572 - Assignment Expressions The goal of the walrus operator is to assign variables within an expression. After completing PEP 572, Guido van Rossum, commonly known as BDFL, decided to resign as a Python dictator. >>> f = (0, 1) >>> [(f := (f[1], sum(f)))[0] for i in range(10)] [1, 1, 2, 3, 5, 8, 13, 21, 34, 55] 1.1.27 Positional-only parameters New in Python 3.8 • PEP 570 - Python Positional-Only Parameters >>> def f(a, b, /, c, d): ... print(a, b, c, d) ... >>> f(1, 2, 3, 4) 1 2 3 4 >>> f(1, 2, c=3, d=4) 1 2 3 4 >>> f(1, b=2, c=3, d=4) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: f() got some positional-only arguments passed as keyword arguments: 'b' 18 Chapter 1. What’s New In Python 3 CHAPTER TWO CHEAT SHEET This part mainly focuses on common snippets in Python code. The cheat sheet not only includes basic Python features but also data structures and algorithms. 2.1 Style Table of Contents • Style – Naming ∗ Class ∗ Function ∗ Variable 2.1.1 Naming Class Bad class fooClass: ... class foo_class: ... Good class FooClass: ... 21 python-cheatsheet Documentation, Release 0.1.0 Function Bad def CapCamelCase(*a): ... def mixCamelCase(*a): ... Good def func_separated_by_underscores(*a): ... Variable Bad FooVar = "CapWords" fooVar = "mixedCase" Foo_Var = "CapWords_With_Underscore" Good # local variable var = "lowercase" # internal use _var = "_single_leading_underscore" # avoid conflicts with Python keyword var_ = "single_trailing_underscore_" # a class attribute (private use in class) __var = " __double_leading_underscore" # "magic" objects or attributes, ex: __init__ __name__ # throwaway variable, ex: _, v = (1, 2) _ = "throwaway" 2.2 From Scratch The main goal of this cheat sheet is to collect some common and basic semantics or snippets. The cheat sheet includes some syntax, which we have already known but still ambiguous in our mind, or some snippets, which we google them again and again. In addition, because the end Of life date for Python 2 is coming. Most of the snippets are mainly based on Python 3’s syntax. Table of Contents • From Scratch – Hello world! 22 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 – Python Version – Ellipsis – if . . . elif . . . else – for Loop – for . . . else . . . – Using range – while . . . else . . . – The do while Statement – try . . . except . . . else . . . – String – List – Dict – Function – Function Annotations – Generators – Generator Delegation – Class – async / await – Avoid exec and eval 2.2.1 Hello world! When we start to learn a new language, we usually learn from printing Hello world!. In Python, we can use another way to print the message by importing __hello__ module. The source code can be found on frozen.c. >>> print("Hello world!") Hello world! >>> import __hello__ Hello world! >>> import __phello__ Hello world! >>> import __phello__.spam Hello world! 2.2. From Scratch 23 python-cheatsheet Documentation, Release 0.1.0 >>> is_break = False >>> for x in range(5): ... if x % 2 == 0: ... is_break = True ... break ... >>> if is_break: ... print("break") ... break >>> for x in range(5): ... if x % 2 == 0: ... print("break") ... break ... else: ... print("no break") ... break 2.2.7 Using range The problem of range in Python 2 is that range may take up a lot of memory if we need to iterate a loop many times. Consequently, using xrange is recommended in Python 2. >>> import platform >>> import sys >>> platform.python_version() '2.7.15' >>> sys.getsizeof(range(100000000)) 800000072 >>> sys.getsizeof(xrange(100000000)) 40 In Python 3, the built-in function range returns an iterable range object instead of a list. The behavior of range is the same as the xrange in Python 2. Therefore, using range do not take up huge memory anymore if we want to run a code block many times within a loop. Further information can be found on PEP 3100. >>> import platform >>> import sys >>> platform.python_version() '3.7.1' >>> sys.getsizeof(range(100000000)) 48 26 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.2.8 while . . . else . . . The else clause belongs to a while loop serves the same purpose as the else clause in a for loop. We can observe that the else does not run when the break occurs in the while loop. >>> n = 0 >>> while n < 5: ... if n == 3: ... break ... n += 1 ... else: ... print("no break") ... 2.2.9 The do while Statement There are many programming languages such as C/C++, Ruby, or Javascript, provide the do while statement. In Python, there is no do while statement. However, we can place the condition and the break at the end of a while loop to achieve the same thing. >>> n = 0 >>> while True: ... n += 1 ... if n == 5: ... break ... >>> n 5 2.2.10 try . . . except . . . else . . . Most of the time, we handle errors in except clause and clean up resources in finally clause. Interestingly, the try statement also provides an else clause for us to avoid catching an exception which was raised by the code that should not be protected by try ... except. The else clause runs when no exception occurs between try and except. >>> try: ... print("No exception") ... except: ... pass ... else: ... print("Success") ... No exception Success 2.2. From Scratch 27 python-cheatsheet Documentation, Release 0.1.0 2.2.11 String Unlike other programming languages, Python does not support string’s item assignment directly. Therefore, if it is necessary to manipulate string’s items, e.g., swap items, we have to convert a string to a list and do a join operation after a series item assignments finish. >>> a = "Hello Python" >>> l = list(a) >>> l[0], l[6] = 'h', 'p' >>> ''.join(l) 'hello python' 2.2.12 List Lists are versatile containers. Python provides a lot of ways such as negative index, slicing statement, or list com- prehension to manipulate lists. The following snippet shows some common operations of lists. >>> a = [1, 2, 3, 4, 5] >>> a[-1] # negative index 5 >>> a[1:] # slicing [2, 3, 4, 5] >>> a[1:-1] [2, 3, 4] >>> a[1:-1:2] [2, 4] >>> a[::-1] # reverse [5, 4, 3, 2, 1] >>> a[0] = 0 # set an item >>> a [0, 2, 3, 4, 5] >>> a.append(6) # append an item >>> a [0, 2, 3, 4, 5, 6] >>> del a[-1] # del an item >>> a [0, 2, 3, 4, 5] >>> b = [x for x in range(3)] # list comprehension >>> b [0, 1, 2] >>> a + b # add two lists [0, 2, 3, 4, 5, 0, 1, 2] 28 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.2.18 Class Python supports many common features such as class documents, multiple inheritance, class variables, instance variables, static method, class method, and so on. Furthermore, Python provides some special methods for program- mers to implement iterators, context manager, etc. The following snippet displays common definition of a class. class A: ... class B: ... class Foo(A, B): """A class document.""" foo = "class variable" def __init__(self, v): self.attr = v self.__private = "private var" @staticmethod def bar_static_method(): ... @classmethod def bar_class_method(cls): ... def bar(self): """A method document.""" def bar_with_arg(self, arg): ... def bar_with_args(self, *args): ... def bar_with_kwarg(self, kwarg="bar"): ... def bar_with_args_kwargs(self, *args, **kwargs): ... def bar_with_kwonly(self, *, k): ... def bar_with_annotations(self, a: int): ... 2.2.19 async / await async and await syntax was introduced from Python 3.5. They were designed to be used with an event loop. Some other features such as the asynchronous generator were implemented in later versions. A coroutine function (async def) are used to create a coroutine for an event loop. Python provides a built-in module, asyncio, to write a concurrent code through async/await syntax. The following snippet shows a simple example of using asyncio. The code must be run on Python 3.7 or above. import asyncio async def http_ok(r, w): head = b"HTTP/1.1 200 OK\r\n" head += b"Content-Type: text/html\r\n" head += b"\r\n" body = b"<html>" body += b"<body><h1>Hello world!</h1></body>" body += b"</html>" (continues on next page) 2.2. From Scratch 31 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) _ = await r.read(1024) w.write(head + body) await w.drain() w.close() async def main(): server = await asyncio.start_server( http_ok, "127.0.0.1", 8888 ) async with server: await server.serve_forever() asyncio.run(main()) 2.2.20 Avoid exec and eval The following snippet shows how to use the built-in function exec. Yet, using exec and eval are not recommended because of some security issues and unreadable code for a human. Further reading can be found on Be careful with exec and eval in Python and Eval really is dangerous >>> py = ''' ... def fib(n): ... a, b = 0, 1 ... for _ in range(n): ... b, a = b + a, b ... return a ... print(fib(10)) ... ''' >>> exec(py, globals(), locals()) 55 2.3 Future Future statements tell the interpreter to compile some semantics as the semantics which will be available in the future Python version. In other words, Python uses from __future__ import feature to backport features from other higher Python versions to the current interpreter. In Python 3, many features such as print_function are already enabled, but we still leave these future statements for backward compatibility. Future statements are NOT import statements. Future statements change how Python interprets the code. They MUST be at the top of the file. Otherwise, Python interpreter will raise SyntaxError. If you’re interested in future statements and want to acquire more explanation, further information can be found on PEP 236 - Back to the __future__ Table of Contents • Future – List All New Features 32 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 – Print Function – Unicode – Division – Annotations – BDFL Retirement – Braces 2.3.1 List All New Features __future__ is a Python module. We can use it to check what kind of future features can import to current Python interpreter. The fun is import __future__ is NOT a future statement, it is a import statement. >>> from pprint import pprint >>> import __future__ >>> pprint(__future__.all_feature_names) ['nested_scopes', 'generators', 'division', 'absolute_import', 'with_statement', 'print_function', 'unicode_literals', 'barry_as_FLUFL', 'generator_stop', 'annotations'] Future statements not only change the behavior of the Python interpreter but also import __future__._Feature into the current program. >>> from __future__ import print_function >>> print_function _Feature((2, 6, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0), 65536) 2.3.2 Print Function Replacing print statement to print function is one of the most notorious decision in Python history. However, this change brings some flexibilities to extend the ability of print. Further information can be found on PEP 3105. >>> print "Hello World" # print is a statement Hello World >>> from __future__ import print_function >>> print "Hello World" File "<stdin>", line 1 print "Hello World" ^ SyntaxError: invalid syntax >>> print("Hello World") # print become a function Hello World 2.3. Future 33 python-cheatsheet Documentation, Release 0.1.0 – String – Characters – Porting unicode(s, ‘utf-8’) – Unicode Code Point – Encoding – Decoding – Unicode Normalization – Avoid UnicodeDecodeError – Long String 2.4.1 String In Python 2, strings are represented in bytes, not Unicode. Python provides different types of string such as Unicode string, raw string, and so on. In this case, if we want to declare a Unicode string, we add u prefix for string literals. >>> s = 'Café' # byte string >>> s 'Caf\xc3\xa9' >>> type(s) <type 'str'> >>> u = u'Café' # unicode string >>> u u'Caf\xe9' >>> type(u) <type 'unicode'> In Python 3, strings are represented in Unicode. If we want to represent a byte string, we add the b prefix for string literals. Note that the early Python versions (3.0-3.2) do not support the u prefix. In order to ease the pain to migrate Unicode aware applications from Python 2, Python 3.3 once again supports the u prefix for string literals. Further information can be found on PEP 414 >>> s = 'Café' >>> type(s) <class 'str'> >>> s 'Café' >>> s.encode('utf-8') b'Caf\xc3\xa9' >>> s.encode('utf-8').decode('utf-8') 'Café' 36 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.4.2 Characters Python 2 takes all string characters as bytes. In this case, the length of strings may be not equivalent to the number of characters. For example, the length of Café is 5, not 4 because é is encoded as a 2 bytes character. >>> s= 'Café' >>> print([_c for _c in s]) ['C', 'a', 'f', '\xc3', '\xa9'] >>> len(s) 5 >>> s = u'Café' >>> print([_c for _c in s]) [u'C', u'a', u'f', u'\xe9'] >>> len(s) 4 Python 3 takes all string characters as Unicode code point. The lenght of a string is always equivalent to the number of characters. >>> s = 'Café' >>> print([_c for _c in s]) ['C', 'a', 'f', 'é'] >>> len(s) 4 >>> bs = bytes(s, encoding='utf-8') >>> print(bs) b'Caf\xc3\xa9' >>> len(bs) 5 2.4.3 Porting unicode(s, ‘utf-8’) The unicode() built-in function was removed in Python 3 so what is the best way to convert the expression unicode(s, 'utf-8') so it works in both Python 2 and 3? In Python 2: >>> s = 'Café' >>> unicode(s, 'utf-8') u'Caf\xe9' >>> s.decode('utf-8') u'Caf\xe9' >>> unicode(s, 'utf-8') == s.decode('utf-8') True In Python 3: >>> s = 'Café' >>> s.decode('utf-8') AttributeError: 'str' object has no attribute 'decode' So, the real answer is. . . 2.4. Unicode 37 python-cheatsheet Documentation, Release 0.1.0 2.4.4 Unicode Code Point ord is a powerful built-in function to get a Unicode code point from a given character. Consequently, If we want to check a Unicode code point of a character, we can use ord. >>> s = u'Café' >>> for _c in s: print('U+%04x' % ord(_c)) ... U+0043 U+0061 U+0066 U+00e9 >>> u = '' >>> for _c in u: print('U+%04x' % ord(_c)) ... U+4e2d U+6587 2.4.5 Encoding A Unicode code point transfers to a byte string is called encoding. >>> s = u'Café' >>> type(s.encode('utf-8')) <class 'bytes'> 2.4.6 Decoding A byte string transfers to a Unicode code point is called decoding. >>> s = bytes('Café', encoding='utf-8') >>> s.decode('utf-8') 'Café' 2.4.7 Unicode Normalization Some characters can be represented in two similar form. For example, the character, é can be written as e (Canonical Decomposition) or é (Canonical Composition). In this case, we may acquire unexpected results when we are comparing two strings even though they look alike. Therefore, we can normalize a Unicode form to solve the issue. # python 3 >>> u1 = 'Café' # unicode string >>> u2 = 'Cafe\u0301' >>> u1, u2 ('Café', 'Cafe') >>> len(u1), len(u2) (4, 5) >>> u1 == u2 False >>> u1.encode('utf-8') # get u1 byte string (continues on next page) 38 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 – New a List – Circular Buffer – Chunk – Groupby – Binary Search – Lower Bound – Upper Bound – Lexicographically Order – Trie 2.5.1 From Scratch There are so many ways that we can manipulate lists in Python. Before we start to learn those versatile manipulations, the following snippet shows the most common operations of lists. >>> a = [1, 2, 3, 4, 5] >>> # contains >>> 2 in a True >>> # positive index >>> a[0] 1 >>> # negative index >>> a[-1] 5 >>> # slicing list[start:end:step] >>> a[1:] [2, 3, 4, 5] >>> a[1:-1] [2, 3, 4] >>> a[1:-1:2] [2, 4] >>> # reverse >>> a[::-1] [5, 4, 3, 2, 1] >>> a[:0:-1] [5, 4, 3, 2] >>> # set an item >>> a[0] = 0 >>> a [0, 2, 3, 4, 5] >>> # append items to list >>> a.append(6) >>> a [0, 2, 3, 4, 5, 6] >>> a.extend([7, 8, 9]) >>> a [0, 2, 3, 4, 5, 6, 7, 8, 9] (continues on next page) 2.5. List 41 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) >>> # delete an item >>> del a[-1] >>> a [0, 2, 3, 4, 5, 6, 7, 8] >>> # list comprehension >>> b = [x for x in range(3)] >>> b [0, 1, 2] >>> # add two lists >>> a + b [0, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2] 2.5.2 Initialize Generally speaking, we can create a list through * operator if the item in the list expression is an immutable object. >>> a = [None] * 3 >>> a [None, None, None] >>> a[0] = "foo" >>> a ['foo', None, None] However, if the item in the list expression is a mutable object, the * operator will copy the reference of the item N times. In order to avoid this pitfall, we should use a list comprehension to initialize a list. >>> a = [[]] * 3 >>> b = [[] for _ in range(3)] >>> a[0].append("Hello") >>> a [['Hello'], ['Hello'], ['Hello']] >>> b[0].append("Python") >>> b [['Python'], [], []] 2.5.3 Copy Assigning a list to a variable is a common pitfall. This assignment does not copy the list to the variable. The variable only refers to the list and increase the reference count of the list. import sys >>> a = [1, 2, 3] >>> sys.getrefcount(a) 2 >>> b = a >>> sys.getrefcount(a) 3 >>> b[2] = 123456 # a[2] = 123456 >>> b [1, 2, 123456] (continues on next page) 42 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) >>> a [1, 2, 123456] There are two types of copy. The first one is called shallow copy (non-recursive copy) and the second one is called deep copy (recursive copy). Most of the time, it is sufficient for us to copy a list by shallow copy. However, if a list is nested, we have to use a deep copy. >>> # shallow copy >>> a = [1, 2] >>> b = list(a) >>> b[0] = 123 >>> a [1, 2] >>> b [123, 2] >>> a = [[1], [2]] >>> b = list(a) >>> b[0][0] = 123 >>> a [[123], [2]] >>> b [[123], [2]] >>> # deep copy >>> import copy >>> a = [[1], [2]] >>> b = copy.deepcopy(a) >>> b[0][0] = 123 >>> a [[1], [2]] >>> b [[123], [2]] 2.5.4 Using slice Sometimes, our data may concatenate as a large segment such as packets. In this case, we will represent the range of data by using slice objects as explaining variables instead of using slicing expressions. >>> icmp = ( ... b"080062988e2100005bff49c20005767c" ... b"08090a0b0c0d0e0f1011121314151617" ... b"18191a1b1c1d1e1f2021222324252627" ... b"28292a2b2c2d2e2f3031323334353637" ... ) >>> head = slice(0, 32) >>> data = slice(32, len(icmp)) >>> icmp[head] b'080062988e2100005bff49c20005767c' 2.5. List 43 python-cheatsheet Documentation, Release 0.1.0 2.5.10 Stacks There is no need for an additional data structure, stack, in Python because the list provides append and pop methods which enable us use a list as a stack. >>> stack = [] >>> stack.append(1) >>> stack.append(2) >>> stack.append(3) >>> stack [1, 2, 3] >>> stack.pop() 3 >>> stack.pop() 2 >>> stack [1] 2.5.11 in Operation We can implement the __contains__ method to make a class do in operations. It is a common way for a programmer to emulate a membership test operations for custom classes. class Stack: def __init__(self): self.__list = [] def push(self, val): self.__list.append(val) def pop(self): return self.__list.pop() def __contains__(self, item): return True if item in self.__list else False stack = Stack() stack.push(1) print(1 in stack) print(0 in stack) Example python stack.py True False 46 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.5.12 Accessing Items Making custom classes perform get and set operations like lists is simple. We can implement a __getitem__ method and a __setitem__ method to enable a class to retrieve and overwrite data by index. In addition, if we want to use the function, len, to calculate the number of elements, we can implement a __len__ method. class Stack: def __init__(self): self.__list = [] def push(self, val): self.__list.append(val) def pop(self): return self.__list.pop() def __repr__(self): return "{}".format(self.__list) def __len__(self): return len(self.__list) def __getitem__(self, idx): return self.__list[idx] def __setitem__(self, idx, val): self.__list[idx] = val stack = Stack() stack.push(1) stack.push(2) print("stack:", stack) stack[0] = 3 print("stack:", stack) print("num items:", len(stack)) Example $ python stack.py stack: [1, 2] stack: [3, 2] num items: 2 2.5. List 47 python-cheatsheet Documentation, Release 0.1.0 2.5.13 Delegating Iterations If a custom container class holds a list and we want iterations to work on the container, we can implement a __iter__ method to delegate iterations to the list. Note that the method, __iter__, should return an iterator object, so we cannot return the list directly; otherwise, Python raises a TypeError. class Stack: def __init__(self): self.__list = [] def push(self, val): self.__list.append(val) def pop(self): return self.__list.pop() def __iter__(self): return iter(self.__list) stack = Stack() stack.push(1) stack.push(2) for s in stack: print(s) Example $ python stack.py 1 2 2.5.14 Sorting Python list provides a built-in list.sort method which sorts a list in-place without using extra memory. Moreover, the return value of list.sort is None in order to avoid confusion with sorted and the function can only be used for list. >>> l = [5, 4, 3, 2, 1] >>> l.sort() >>> l [1, 2, 3, 4, 5] >>> l.sort(reverse=True) >>> l [5, 4, 3, 2, 1] The sorted function does not modify any iterable object in-place. Instead, it returns a new sorted list. Using sorted is safer than list.sort if some list’s elements are read-only or immutable. Besides, another difference between list.sort and sorted is that sorted accepts any iterable object. >>> l = [5, 4, 3, 2, 1] >>> new = sorted(l) >>> new (continues on next page) 48 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) ... >>> nodes = [Node(3), Node(2), Node(1)] >>> nodes.sort(key=cmp_to_key(lambda x,y: x.val - y.val)) >>> nodes [Node(1), Node(2), Node(3)] 2.5.15 Sorted List import bisect class Foo(object): def __init__(self, k): self.k = k def __eq__(self, rhs): return self.k == rhs.k def __ne__(self, rhs): return self.k != rhs.k def __lt__(self, rhs): return self.k < rhs.k def __gt__(self, rhs): return self.k > rhs.k def __le__(self, rhs): return self.k <= rhs.k def __ge__(self, rhs): return self.k >= rhs.k def __repr__(self): return f"Foo({self.k})" def __str__(self): return self.__repr__() foo = [Foo(1), Foo(3), Foo(2), Foo(0)] bar = [] for x in foo: bisect.insort(bar, x) print(bar) # [Foo(0), Foo(1), Foo(2), Foo(3)] 2.5. List 51 python-cheatsheet Documentation, Release 0.1.0 2.5.16 New a List # new a list with size = 3 >>> [0] * 3 [0, 0, 0] # new a 2d list with size 3x3 >>> [[0] * 3 for _ in range(3)] [[0, 0, 0], [0, 0, 0], [0, 0, 0]] Note that we should avoid creating a multi-dimension list via the following snippet because all objects in the list point to the same address. >>> a = [[0] * 3] * 3 >>> a [[0, 0, 0], [0, 0, 0], [0, 0, 0]] >>> a[1][1] = 2 >>> a [[0, 2, 0], [0, 2, 0], [0, 2, 0]] 2.5.17 Circular Buffer >>> from collections import deque >>> d = deque(maxlen=8) >>> for x in range(9): ... d.append(x) ... >>> d deque([1, 2, 3, 4, 5, 6, 7, 8], maxlen=8) >>> from collections import deque >>> def tail(path, n=10): ... with open(path) as f: ... return deque(f, n) ... >>> tail("/etc/hosts") 2.5.18 Chunk >>> def chunk(lst, n): ... for i in range(0, len(lst), n): ... yield lst[i:i+n] ... >>> a = [1, 2, 3, 4, 5, 6, 7, 8] >>> list(chunk(a, 3)) [[1, 2, 3], [4, 5, 6], [7, 8]] 52 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.5.19 Groupby >>> import itertools >>> s = "AAABBCCCCC" >>> for k, v in itertools.groupby(s): ... print(k, list(v)) ... A ['A', 'A', 'A'] B ['B', 'B'] C ['C', 'C', 'C', 'C', 'C'] # group by key >>> x = [('gp1', 'a'), ('gp2', 'b'), ('gp2', 'c')] >>> for k, v in itertools.groupby(x, lambda x: x[0]): ... print(k, list(v)) ... gp1 [('gp1', 'a')] gp2 [('gp2', 'b'), ('gp2', 'c')] 2.5.20 Binary Search >>> def binary_search(arr, x, lo=0, hi=None): ... if not hi: hi = len(arr) ... pos = bisect_left(arr, x, lo, hi) ... return pos if pos != hi and arr[pos] == x else -1 ... >>> a = [1, 1, 1, 2, 3] >>> binary_search(a, 1) 0 >>> binary_search(a, 2) 3 2.5.21 Lower Bound >>> import bisect >>> a = [1,2,3,3,4,5] >>> bisect.bisect_left(a, 3) 2 >>> bisect.bisect_left(a, 3.5) 4 2.5. List 53 python-cheatsheet Documentation, Release 0.1.0 2.6.2 Uniquify a List >>> a = [1, 2, 2, 2, 3, 4, 5, 5] >>> a [1, 2, 2, 2, 3, 4, 5, 5] >>> ua = list(set(a)) >>> ua [1, 2, 3, 4, 5] 2.6.3 Union Two Sets >>> a = set([1, 2, 2, 2, 3]) >>> b = set([5, 5, 6, 6, 7]) >>> a | b set([1, 2, 3, 5, 6, 7]) >>> # or >>> a = [1, 2, 2, 2, 3] >>> b = [5, 5, 6, 6, 7] >>> set(a + b) set([1, 2, 3, 5, 6, 7]) 2.6.4 Append Items to a Set >>> a = set([1, 2, 3, 3, 3]) >>> a.add(5) >>> a set([1, 2, 3, 5]) >>> # or >>> a = set([1, 2, 3, 3, 3]) >>> a |= set([1, 2, 3, 4, 5, 6]) >>> a set([1, 2, 3, 4, 5, 6]) 2.6.5 Intersection Two Sets >>> a = set([1, 2, 2, 2, 3]) >>> b = set([1, 5, 5, 6, 6, 7]) >>> a & b set([1]) 56 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.6.6 Common Items from Sets >>> a = [1, 1, 2, 3] >>> b = [1, 3, 5, 5, 6, 6] >>> com = list(set(a) & set(b)) >>> com [1, 3] 2.6.7 Contain b contains a >>> a = set([1, 2]) >>> b = set([1, 2, 5, 6]) >>> a <= b True a contains b >>> a = set([1, 2, 5, 6]) >>> b = set([1, 5, 6]) >>> a >= b True 2.6.8 Set Diff >>> a = set([1, 2, 3]) >>> b = set([1, 5, 6, 7, 7]) >>> a - b set([2, 3]) 2.6.9 Symmetric diff >>> a = set([1,2,3]) >>> b = set([1, 5, 6, 7, 7]) >>> a ^ b set([2, 3, 5, 6, 7]) 2.7 Dictionary Table of Contents • Dictionary – Get All Keys 2.7. Dictionary 57 python-cheatsheet Documentation, Release 0.1.0 – Get Key and Value – Find Same Keys – Set a Default Value – Update Dictionary – Merge Two Dictionaries – Emulating a Dictionary – LRU Cache 2.7.1 Get All Keys >>> a = {"1":1, "2":2, "3":3} >>> b = {"2":2, "3":3, "4":4} >>> a.keys() ['1', '3', '2'] 2.7.2 Get Key and Value >>> a = {"1":1, "2":2, "3":3} >>> a.items() 2.7.3 Find Same Keys >>> a = {"1":1, "2":2, "3":3} >>> b = {"2":2, "3":3, "4":4} >>> [_ for _ in a.keys() if _ in b.keys()] ['3', '2'] >>> # better way >>> c = set(a).intersection(set(b)) >>> list(c) ['3', '2'] >>> # or >>> [_ for _ in a if _ in b] ['3', '2'] [('1', 1), ('3', 3), ('2', 2)] 58 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) ... print() ... 1 3 5 >>> '1' in emud # __contains__ True 2.7.8 LRU Cache from collections import OrderedDict class LRU(object): def __init__(self, maxsize=128): self._maxsize = maxsize self._cache = OrderedDict() def get(self, k): if k not in self._cache: return None self._cache.move_to_end(k) return self._cache[k] def put(self, k, v): if k in self._cache: self._cache.move_to_end(k) self._cache[k] = v if len(self._cache) > self._maxsize: self._cache.popitem(last=False) def __str__(self): return str(self._cache) def __repr__(self): return self.__str__() Note that dictionaries preserve insertion order from Python 3.7. Moreover, updating a key does not affect the order. Therefore, a dictionary can also simulate an LRU cache, which is similar to using an OrderedDict. class LRU(object): def __init__(self, maxsize=128): self._maxsize = maxsize self._cache = {} def get(self, k): if k not in self._cache: return None self.move_to_end(k) return self._cache[k] (continues on next page) 2.7. Dictionary 61 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) def put(self, k, v): if k in self._cache: self.move_to_end(k) self._cache[k] = v if len(self._cache) > self._maxsize: self.pop() def pop(self): it = iter(self._cache.keys()) del self._cache[next(it)] def move_to_end(self, k): if k not in self._cache: return v = self._cache[k] del self._cache[k] self._cache[k] = v def __str__(self): return str(self._cache) def __repr__(self): return self.__str__() 2.8 Heap Table of Contents • Heap – Heap Sort – Priority Queue 2.8.1 Heap Sort >>> import heapq >>> a = [5, 1, 3, 2, 6] >>> h = [] >>> for x in a: ... heapq.heappush(h, x) ... >>> x = [heapq.heappop(h) for _ in range(len(a))] >>> x [1, 2, 3, 5, 6] 62 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.8.2 Priority Queue import heapq h = [] heapq.heappush(h, (1, "1")) # (priority, value) heapq.heappush(h, (5, "5")) heapq.heappush(h, (3, "3")) heapq.heappush(h, (2, "2")) x = [heapq.heappop(h) for _ in range(len(h))] print(x) import heapq class Num(object): def __init__(self, n): self._n = n def __lt__(self, other): return self._n < other._n def __repr__(self): return self.__str__() def __str__(self): return f"Num({self._n})" h = [] heapq.heappush(h, Num(5)) heapq.heappush(h, Num(2)) heapq.heappush(h, Num(1)) x = [heapq.heappop(h) for _ in range(len(h))] print(x) # output: [Num(1), Num(2), Num(5)] 2.9 Function A function can help programmers to wrap their logic into a task for avoiding duplicate code. In Python, the definition of a function is so versatile that we can use many features such as decorator, annotation, docstrings, default arguments and so on to define a function. In this cheat sheet, it collects many ways to define a function and demystifies some enigmatic syntax in functions. Table of Contents • Function – Document Functions – Default Arguments – Option Arguments 2.9. Function 63 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) >>> fib(10) 55 >>> fib.__annotations__ {'n': <class 'int'>, 'return': <class 'int'>} 2.9.7 Callable In some cases such as passing a callback function, we need to check whether an object is callable or not. The built-in function, callable, assist us to avoid raising a TypeError if the object is not callable. >>> a = 10 >>> def fun(): ... print("I am callable") ... >>> callable(a) False >>> callable(fun) True 2.9.8 Get Function Name >>> def example_function(): ... pass ... >>> example_function.__name__ 'example_function' 2.9.9 Lambda Sometimes, we don’t want to use the def statement to define a short callback function. We can use a lambda expression as a shortcut to define an anonymous or an inline function instead. However, only one single expression can be specified in lambda. That is, no other features such as multi-line statements, conditions, or exception handling can be contained. >>> fn = lambda x: x**2 >>> fn(3) 9 >>> (lambda x: x**2)(3) 9 >>> (lambda x: [x*_ for _ in range(5)])(2) [0, 2, 4, 6, 8] >>> (lambda x: x if x>3 else 3)(5) 5 66 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.9.10 Generator >>> def fib(n): ... a, b = 0, 1 ... for _ in range(n): ... yield a ... b, a = a + b, b ... >>> [f for f in fib(10)] [0, 1, 1, 2, 3, 5, 8, 13, 21, 34] 2.9.11 Decorator New in Python 2.4 • PEP 318 - Decorators for Functions and Methods >>> from functools import wraps >>> def decorator(func): ... @wraps(func) ... def wrapper(*args, **kwargs): ... print("Before calling {}.".format(func.__name__)) ... ret = func(*args, **kwargs) ... print("After calling {}.".format(func.__name__)) ... return ret ... return wrapper ... >>> @decorator ... def example(): ... print("Inside example function.") ... >>> example() Before calling example. Inside example function. After calling example. Equals to ... def example(): ... print("Inside example function.") ... >>> example = decorator(example) >>> example() Before calling example. Inside example function. After calling example. 2.9. Function 67 python-cheatsheet Documentation, Release 0.1.0 2.9.12 Decorator with Arguments >>> from functools import wraps >>> def decorator_with_argument(val): ... def decorator(func): ... @wraps(func) ... def wrapper(*args, **kwargs): ... print("Val is {0}".format(val)) ... return func(*args, **kwargs) ... return wrapper ... return decorator ... >>> @decorator_with_argument(10) ... def example(): ... print("This is example function.") ... >>> example() Val is 10 This is example function. Equals to >>> def example(): ... print("This is example function.") ... >>> example = decorator_with_argument(10)(example) >>> example() Val is 10 This is example function. 2.9.13 Cache New in Python 3.2 Without Cache >>> import time >>> def fib(n): ... if n < 2: ... return n ... return fib(n - 1) + fib(n - 2) ... >>> s = time.time(); _ = fib(32); e = time.time(); e - s 1.1562161445617676 With Cache (dynamic programming) >>> from functools import lru_cache >>> @lru_cache(maxsize=None) ... def fib(n): ... if n < 2: ... return n ... return fib(n - 1) + fib(n - 2) (continues on next page) 68 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.10.4 Has / Get / Set Attributes >>> class Example(object): ... def __init__(self): ... self.name = "ex" ... def printex(self): ... print("This is an example") ... >>> ex = Example() >>> hasattr(ex,"name") True >>> hasattr(ex,"printex") True >>> hasattr(ex,"print") False >>> getattr(ex,'name') 'ex' >>> setattr(ex,'name','example') >>> ex.name 'example' 2.10.5 Check Inheritance >>> class Example(object): ... def __init__(self): ... self.name = "ex" ... def printex(self): ... print("This is an Example") ... >>> issubclass(Example, object) True 2.10.6 Get Class Name >>> class ExampleClass(object): ... pass ... >>> ex = ExampleClass() >>> ex.__class__.__name__ 'ExampleClass' 2.10. Classes and Objects 71 python-cheatsheet Documentation, Release 0.1.0 2.10.7 New and Init __init__ will be invoked >>> class ClassA(object): ... def __new__(cls, arg): ... print('__new__ ' + arg) ... return object.__new__(cls, arg) ... def __init__(self, arg): ... print('__init__ ' + arg) ... >>> o = ClassA("Hello") __new__ Hello __init__ Hello __init__ won’t be invoked >>> class ClassB(object): ... def __new__(cls, arg): ... print('__new__ ' + arg) ... return object ... def __init__(self, arg): ... print('__init__ ' + arg) ... >>> o = ClassB("Hello") __new__ Hello 2.10.8 The Diamond Problem The problem of multiple inheritance in searching a method >>> def foo_a(self): ... print("This is ClsA") ... >>> def foo_b(self): ... print("This is ClsB") ... >>> def foo_c(self): ... print("This is ClsC") ... >>> class Type(type): ... def __repr__(cls): ... return cls.__name__ ... >>> ClsA = Type("ClsA", (object,), {'foo': foo_a}) >>> ClsB = Type("ClsB", (ClsA,), {'foo': foo_b}) >>> ClsC = Type("ClsC", (ClsA,), {'foo': foo_c}) >>> ClsD = Type("ClsD", (ClsB, ClsC), {}) >>> ClsD.mro() [ClsD, ClsB, ClsC, ClsA, <type 'object'>] >>> ClsD().foo() This is ClsB 72 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.10.9 Representation of a Class >>> class Example(object): ... def __str__(self): ... return "Example __str__" ... def __repr__(self): ... return "Example __repr__" ... >>> print(str(Example())) Example __str__ >>> Example() Example __repr__ 2.10.10 Callable Object >>> class CallableObject(object): ... def example(self, *args, **kwargs): ... print("I am callable!") ... def __call__(self, *args, **kwargs): ... self.example(*args, **kwargs) ... >>> ex = CallableObject() >>> ex() I am callable! 2.10.11 Context Manager # replace try: ... finally: ... # see: PEP343 # common use in open and close import socket class Socket(object): def __init__(self, host, port): self.host = host self.port = port def __enter__(self): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.bind((self.host,self.port)) sock.listen(5) self.sock = sock return self.sock def __exit__(self,*exc_info): if exc_info[0] is not None: import traceback traceback.print_exception(*exc_info) self.sock.close() (continues on next page) 2.10. Classes and Objects 73 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) ... inst.__dict__[self._name] = value ... def __delete__(self,inst): ... del inst.__dict__[self._name] ... >>> class Example(object): ... x = Integer('x') ... def __init__(self, val): ... self.x = val ... >>> ex1 = Example(1) >>> ex1.x 1 >>> ex2 = Example("str") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 4, in __init__ File "<stdin>", line 11, in __set__ TypeError: Expected an int >>> ex3 = Example(3) >>> hasattr(ex3, 'x') True >>> del ex3.x >>> hasattr(ex3, 'x') False 2.10.16 Singleton Decorator Singleton is a design pattern that restricts the creation of instances of a class so that it only creates one instance of the class that implements it. #!/usr/bin/env python3 """Singleton decorator class.""" class Singleton(object): def __init__(self, cls): self.__cls = cls self.__obj = None def __call__(self, *args, **kwargs): if not self.__obj: self.__obj = self.__cls(*args, **kwargs) return self.__obj if __name__ == "__main__": # Testing ... @Singleton class Test(object): def __init__(self, text): (continues on next page) 76 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) self.text = text a = Test("Hello") b = Test("World") print("id(a):", id(a), "id(b):", id(b), "Diff:", id(a)-id(b)) 2.10.17 Static and Class Methond @classmethod is bound to a class. @staticmethod is similar to a python function but define in a class. >>> class example(object): ... @classmethod ... def clsmethod(cls): ... print("I am classmethod") ... @staticmethod ... def stmethod(): ... print("I am staticmethod") ... def instmethod(self): ... print("I am instancemethod") ... >>> ex = example() >>> ex.clsmethod() I am classmethod >>> ex.stmethod() I am staticmethod >>> ex.instmethod() I am instancemethod >>> example.clsmethod() I am classmethod >>> example.stmethod() I am staticmethod >>> example.instmethod() Traceback (most recent call last): File "", line 1, in TypeError: unbound method instmethod() ... 2.10.18 Abstract Method abc is used to define methods but not implement >>> from abc import ABCMeta, abstractmethod >>> class base(object): ... __metaclass__ = ABCMeta ... @abstractmethod ... def absmethod(self): ... """ Abstract method """ ... >>> class example(base): ... def absmethod(self): (continues on next page) 2.10. Classes and Objects 77 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) ... print("abstract") ... >>> ex = example() >>> ex.absmethod() abstract Another common way is to raise NotImplementedError >>> class base(object): ... def absmethod(self): ... raise NotImplementedError ... >>> class example(base): ... def absmethod(self): ... print("abstract") ... >>> ex = example() >>> ex.absmethod() abstract 2.10.19 Using slot to Save Memory #!/usr/bin/env python3 import resource import platform import functools def profile_mem(func): @functools.wraps(func) def wrapper(*a, **k): s = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss ret = func(*a, **k) e = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss uname = platform.system() if uname == "Linux": print(f"mem usage: {e - s} kByte") elif uname == "Darwin": print(f"mem usage: {e - s} Byte") else: raise Exception("not support") return ret return wrapper class S(object): __slots__ = ['attr1', 'attr2', 'attr3'] (continues on next page) 78 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 – Unpacking Generators – Implement Iterable object via generator – Send message to generator – yield from expression – yield (from) EXPR return RES – Generate sequences – What RES = yield from EXP actually do? – for _ in gen() simulate yield from – Check generator type – Check Generator State – Simple compiler – Context manager and generator – What @contextmanager actually doing? – profile code block – yield from and __iter__ – yield from == await expression – Closure in Python - using generator – Implement a simple scheduler – Simple round-robin with blocking – simple round-robin with blocking and non-blocking – Asynchronous Generators – Asynchronous generators can have try..finally blocks – send value and throw exception into async generator – Simple async round-robin – Async generator get better performance than async iterator – Asynchronous Comprehensions 2.11.1 Glossary of Generator # generator function >>> def gen_func(): ... yield 5566 ... >>> gen_func <function gen_func at 0x1019273a> # generator (continues on next page) 2.11. Generator 81 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) # # calling the generator function returns a generator >>> g = gen_func() >>> g <generator object gen_func at 0x101238fd> >>> next(g) 5566 >>> next(g) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration # generator expression # # generator expression evaluating directly to a generator >>> g = (x for x in range(2)) >>> g <generator object <genexpr> at 0x10a9c191> >>> next(g) 0 >>> next(g) 1 >>> next(g) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration 2.11.2 Produce value via generator >>> from __future__ import print_function >>> def prime(n): ... p = 2 ... while n > 0: ... for x in range(2, p): ... if p % x == 0: ... break ... else: ... yield p ... n -= 1 ... p += 1 ... >>> p = prime(3) >>> next(p) 2 >>> next(p) 3 >>> next(p) 5 (continues on next page) 82 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) >>> next(p) Traceback (most recent call last): File "<stdin>", line 1, in <module> StopIteration >>> for x in prime(5): ... print(x, end=" ") ... 2 3 5 7 11 >>> 2.11.3 Unpacking Generators # PEP 448 # unpacking inside a list >>> g1 = (x for x in range(3)) >>> g2 = (x**2 for x in range(2)) >>> [1, *g1, 2, *g2] [1, 0, 1, 2, 2, 0, 1] >>> # equal to >>> g1 = (x for x in range(3)) >>> g2 = (x**2 for x in range(2)) >>> [1] + list(g1) + [2] + list(g2) [1, 0, 1, 2, 2, 0, 1] # unpacking inside a set >>> g = (x for x in [5, 5, 6, 6]) >>> {*g} {5, 6} # unpacking to variables >>> g = (x for x in range(3)) >>> a, b, c = g >>> print(a, b, c) 0 1 2 >>> g = (x for x in range(6)) >>> a, b, *c, d = g >>> print(a, b, d) 0 1 5 >>> print(c) [2, 3, 4] # unpacking inside a function >>> print(*(x for x in range(3))) 0 1 2 2.11. Generator 83 python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) ... break ... total += val ... count += 1 ... avg = total / count ... return avg ... >>> g = average() >>> next(g) # start gen >>> g.send(3) >>> g.send(5) >>> try: ... g.send(None) ... except StopIteration as e: ... ret = e.value ... >>> ret 4.0 # yield from EXP return RES >>> def subgen(): ... yield 9527 ... >>> def delegating_gen(): ... yield from subgen() ... return 5566 ... >>> try: ... g = delegating_gen() ... next(g) ... next(g) ... except StopIteration as _e: ... print(_e.value) ... 9527 5566 2.11.8 Generate sequences # get a list via generator >>> def chain(): ... for x in 'ab': ... yield x ... for x in range(3): ... yield x ... >>> a = list(chain()) >>> a ['a', 'b', 0, 1, 2] (continues on next page) 86 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) # equivalent to >>> def chain(): ... yield from 'ab' ... yield from range(3) ... >>> a = list(chain()) >>> a ['a', 'b', 0, 1, 2] 2.11.9 What RES = yield from EXP actually do? # ref: pep380 >>> def subgen(): ... for x in range(3): ... yield x ... >>> EXP = subgen() >>> def delegating_gen(): ... _i = iter(EXP) ... try: ... _y = next(_i) ... except StopIteration as _e: ... RES = _e.value ... else: ... while True: ... _s = yield _y ... try: ... _y = _i.send(_s) ... except StopIteration as _e: ... RES = _e.value ... break ... >>> g = delegating_gen() >>> next(g) 0 >>> next(g) 1 >>> next(g) 2 # equivalent to >>> EXP = subgen() >>> def delegating_gen(): ... RES = yield from EXP ... >>> g = delegating_gen() >>> next(g) 0 >>> next(g) 1 2.11. Generator 87 python-cheatsheet Documentation, Release 0.1.0 2.11.10 for _ in gen() simulate yield from >>> def subgen(n): ... for x in range(n): yield x ... >>> def gen(n): ... yield from subgen(n) ... >>> g = gen(3) >>> next(g) 0 >>> next(g) 1 # equal to >>> def gen(n): ... for x in subgen(n): yield x ... >>> g = gen(3) >>> next(g) 0 >>> next(g) 1 2.11.11 Check generator type >>> from types import GeneratorType >>> def gen_func(): ... yield 5566 ... >>> g = gen_func() >>> isinstance(g, GeneratorType) True >>> isinstance(123, GeneratorType) False 2.11.12 Check Generator State >>> import inspect >>> def gen_func(): ... yield 9527 ... >>> g = gen_func() >>> inspect.getgeneratorstate(g) 'GEN_CREATED' >>> next(g) 9527 >>> inspect.getgeneratorstate(g) 'GEN_SUSPENDED' (continues on next page) 88 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 (continued from previous page) elif node.op == '-': return leftval - rightval elif node.op == '*': return leftval * rightval elif node.op == '/': return leftval / rightval def evaluate(exp): toks = tokenize(exp) tree = parse(toks) return Evaluator().visit(tree) exp = '2 * 3 + 5 / 2' print(evaluate(exp)) exp = '+'.join([str(x) for x in range(10000)]) print(evaluate(exp)) output: python3 compiler.py 8.5 49995000 2.11.14 Context manager and generator >>> import contextlib >>> @contextlib.contextmanager ... def mylist(): ... try: ... l = [1, 2, 3, 4, 5] ... yield l ... finally: ... print("exit scope") ... >>> with mylist() as l: ... print(l) ... [1, 2, 3, 4, 5] exit scope 2.11. Generator 91 python-cheatsheet Documentation, Release 0.1.0 2.11.15 What @contextmanager actually doing? # ref: PyCon 2014 - David Beazley # define a context manager class class GeneratorCM(object): def __init__(self, gen): self._gen = gen def __enter__(self): return next(self._gen) def __exit__(self, *exc_info): try: if exc_info[0] is None: next(self._gen) else: self._gen.throw(*exc_info) raise RuntimeError except StopIteration: return True except: raise # define a decorator def contextmanager(func): def run(*a, **k): return GeneratorCM(func(*a, **k)) return run # example of context manager @contextmanager def mylist(): try: l = [1, 2, 3, 4, 5] yield l finally: print("exit scope") with mylist() as l: print(l) output: $ python ctx.py [1, 2, 3, 4, 5] exit scope 92 Chapter 2. Cheat Sheet python-cheatsheet Documentation, Release 0.1.0 2.11.16 profile code block >>> import time >>> @contextmanager ... def profile(msg): ... try: ... s = time.time() ... yield ... finally: ... e = time.time() ... print('{} cost time: {}'.format(msg, e - s)) ... >>> with profile('block1'): ... time.sleep(1) ... block1 cost time: 1.00105595589 >>> with profile('block2'): ... time.sleep(3) ... block2 cost time: 3.00104284286 2.11.17 yield from and __iter__ >>> class FakeGen: ... def __iter__(self): ... n = 0 ... while True: ... yield n ... n += 1 ... def __reversed__(self): ... n = 9527 ... while True: ... yield n ... n -= 1 ... >>> def spam(): ... yield from FakeGen() ... >>> s = spam() >>> next(s) 0 >>> next(s) 1 >>> next(s) 2 >>> next(s) 3 >>> def reversed_spam(): ... yield from reversed(FakeGen()) ... >>> g = reversed_spam() (continues on next page) 2.11. Generator 93