{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Chapter 2 — An Array of Sequences\n", "\n", "**Sections with code snippets in this chapter:**\n", "\n", "* [List Comprehensions and Generator Expressions](#List-Comprehensions-and-Generator-Expressions)\n", "* [Tuples Are Not Just Immutable Lists](#Tuples-Are-Not-Just-Immutable-Lists)\n", "* [Unpacking sequences and iterables](#Unpacking-sequences-and-iterables)\n", "* [Pattern Matching with Sequences](#Pattern-Matching-with-Sequences)\n", "* [Slicing](#Slicing)\n", "* [Using + and * with Sequences](#Using-+-and-*-with-Sequences)\n", "* [Augmented Assignment with Sequences](#Augmented-Assignment-with-Sequences)\n", "* [list.sort and the sorted Built-In Function](#list.sort-and-the-sorted-Built-In-Function)\n", "* [When a List Is Not the Answer](#When-a-List-Is-Not-the-Answer)\n", "* [Memory Views](#Memory-Views)\n", "* [NumPy and SciPy](#NumPy-and-SciPy)\n", "* [Deques and Other Queues](#Deques-and-Other-Queues)\n", "* [Soapbox](#Soapbox)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Pythonic\n", "- ABC introduced many ideas we now consider “Pythonic”: generic operations on different types\n", "of sequences, built-in tuple and mapping types, structure by indentation, strong\n", "typing without variable declarations, and more\n", "- Python inherited from ABC the uniform handling of sequences. Strings, lists, byte\n", "sequences, arrays, XML elements, and database results share a rich set of common\n", "operations, including iteration, slicing, sorting, and concatenation\n", "- Understanding the variety of sequences available in Python saves us from reinventing\n", "the wheel, and their common interface inspires us to create APIs that properly sup‐\n", "port and leverage existing and future sequence type" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Overview of built-in Sequences" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The standard library offers a rich selection of sequence types implemented in C:\n", "- Container sequences: \n", " - Can hold items of different types, including nested containers. Some examples: list, tuple, and collections.deque.\n", " - A container sequence holds references to the objects it contains, which may be of any type, each item is a separate Python object, possibly holding references to other Python objects\n", "- Flat sequences: \n", " - Hold items of one simple type. Some examples: str, bytes, and array.array\n", " - a flat sequence stores the value of its contents in its own memory space, not as distinct Python objects\n", " - flat sequences are more compact, but they are limited to holding primitive machine values like bytes, integers, and floats.\n", "\n", "Another way of grouping sequence types is by mutability:\n", "- Mutable sequences\n", " - For example, list, bytearray, array.array, and collections.deque.\n", "- Immutable sequences\n", " - For example, tuple, str, and bytes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## List Comprehensions and Generator Expressions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-1. Build a list of Unicode codepoints from a string" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[36, 162, 163, 165, 8364, 164]" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "symbols = '$¢£¥€¤'\n", "codes = []\n", "\n", "for symbol in symbols:\n", " codes.append(ord(symbol))\n", "\n", "codes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-2. Build a list of Unicode codepoints from a string, using a listcomp" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[36, 162, 163, 165, 8364, 164]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "symbols = '$¢£¥€¤'\n", "\n", "codes = [ord(symbol) for symbol in symbols]\n", "\n", "codes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Box: Listcomps No Longer Leak Their Variables" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'ABC'" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = 'ABC'\n", "codes = [ord(x) for x in x]\n", "x" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[65, 66, 67]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "codes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Local Scope Within Comprehensions and Generator Expressions\n", "- In Python 3, list comprehensions, generator expressions, and their siblings set and\n", "dict comprehensions, have a local scope to hold the variables assigned in the for\n", "clause.\n", "- However, variables assigned with the **“Walrus operator” :=** remain accessible after\n", "those comprehensions or expressions return—unlike local variables in a function. The scope of the target of := as the enclosing function, unless there is a global or nonlocal declaration for that target." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "ename": "NameError", "evalue": "name 'c' is not defined", "output_type": "error", "traceback": [ "\u001b[31m---------------------------------------------------------------------------\u001b[39m", "\u001b[31mNameError\u001b[39m Traceback (most recent call last)", "\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[11]\u001b[39m\u001b[32m, line 2\u001b[39m\n\u001b[32m 1\u001b[39m codes = [last := \u001b[38;5;28mord\u001b[39m(c) \u001b[38;5;28;01mfor\u001b[39;00m c \u001b[38;5;129;01min\u001b[39;00m x]\n\u001b[32m----> \u001b[39m\u001b[32m2\u001b[39m \u001b[43mc\u001b[49m\n", "\u001b[31mNameError\u001b[39m: name 'c' is not defined" ] } ], "source": [ "codes = [last := ord(c) for c in x]\n", "c # c has local scope within listcomp" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "67" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "last # last is assigned by \"Walrus operator :=\", it has no local scope" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Listcomps Versus map and filter\n", "- Listcomps do everything the map and filter functions do, without the contortions of the functionally challenged Python lambda" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-3. The same list built by a listcomp and a map/filter composition" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[162, 163, 165, 8364, 164]" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "symbols = '$¢£¥€¤'\n", "beyond_ascii = [ord(s) for s in symbols if ord(s) > 127]\n", "beyond_ascii" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[162, 163, 165, 8364, 164]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "beyond_ascii = list(filter(lambda c: c > 127, map(ord, symbols)))\n", "beyond_ascii" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-4. Cartesian product using a list comprehension" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('black', 'S'),\n", " ('black', 'M'),\n", " ('black', 'L'),\n", " ('white', 'S'),\n", " ('white', 'M'),\n", " ('white', 'L')]" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "colors = ['black', 'white']\n", "sizes = ['S', 'M', 'L']\n", "tshirts = [(color, size) for color in colors for size in sizes]\n", "tshirts" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "('black', 'S')\n", "('black', 'M')\n", "('black', 'L')\n", "('white', 'S')\n", "('white', 'M')\n", "('white', 'L')\n" ] } ], "source": [ "for color in colors:\n", " for size in sizes:\n", " print((color, size))" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('black', 'S'),\n", " ('black', 'M'),\n", " ('black', 'L'),\n", " ('white', 'S'),\n", " ('white', 'M'),\n", " ('white', 'L')]" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "shirts = [(color, size) for size in sizes\n", " for color in colors]\n", "tshirts" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Generator Expressions\n", "- To initialize tuples, arrays, and other types of sequences, you could also start from a\n", "listcomp, but a genexp (generator expression) saves memory because it yields items\n", "one by one using the iterator protocol instead of building a whole list just to feed\n", "another constructor.\n", "- Genexps use the same syntax as listcomps, but are enclosed in parentheses rather\n", "than brackets" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-5. Initializing a tuple and an array from a generator expression" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(36, 162, 163, 165, 8364, 164)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "symbols = '$¢£¥€¤'\n", "tuple(ord(symbol) for symbol in symbols)" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array('I', [36, 162, 163, 165, 8364, 164])" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import array\n", "\n", "array.array('I', (ord(symbol) for symbol in symbols))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-6. Cartesian product in a generator expression" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "black S\n", "black M\n", "black L\n", "white S\n", "white M\n", "white L\n" ] } ], "source": [ "colors = ['black', 'white']\n", "sizes = ['S', 'M', 'L']\n", "\n", "for tshirt in ('%s %s' % (c, s) for c in colors for s in sizes):\n", " print(tshirt)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "## Tuples Are Not Just Immutable Lists" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "tuple has double duty:\n", "- as immutable lists\n", "- as records with no field names" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "#### Example 2-7. Tuples used as records" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When using a tuple as a collection of fields, the number of items is usually fixed and their order is always important" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "BRA/CE342567\n", "ESP/XDA205856\n", "USA/31195855\n" ] } ], "source": [ "lax_coordinates = (33.9425, -118.408056)\n", "city, year, pop, chg, area = ('Tokyo', 2003, 32_450, 0.66, 8014) # tuple unpacking\n", "traveler_ids = [('USA', '31195855'), ('BRA', 'CE342567'), ('ESP', 'XDA205856')]\n", "\n", "for passport in sorted(traveler_ids): # passport is bound to each tuple.\n", " # the % operator assigned each item in the passport tuple to the corresponding slot in the format string is also tuple unpacking\n", " print('%s/%s' % passport) # The % formatting operator understands tuples and treats each item as a separate field" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "USA\n", "BRA\n", "ESP\n" ] } ], "source": [ "# The for loop knows how to retrieve the items of a tuple separately—this is called “tuple unpacking.”\n", "for country, _ in traveler_ids:\n", " print(country)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "### Tuples as Immutable Lists" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Python interpreter and standard library make extensive use of tuples as immutable lists, and so should you. This brings two key benefits:\n", "- Clarity:\n", " - When you see a tuple in code, you know its length will never change\n", "- Performance\n", " - A tuple uses less memory than a list of the same length, and it allows Python to do some optimizations\n", "\n", "References immutable\n", "- However, be aware that the immutability of a tuple only applies to the references\n", "contained in it. References in a tuple cannot be deleted or replaced. But if one of\n", "those references points to a mutable object, and that object is changed, then the value of the tuple changes" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = (10, 'alpha', [1, 2])\n", "b = (10, 'alpha', [1, 2])\n", "a == b" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b[-1].append(99) # [1,2] is list object , can be modified\n", "a == b" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(10, 'alpha', [1, 2, 99])" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### What is Hashable\n", "- Tuples with mutable items can be a source of bugs.\n", "- An object is only hashable if its value cannot ever change. An unhashable tuple cannot be inserted as a dict key, or a set element" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# If you want to determine explicitly if a tuple (or any object) has a fixed value, you can\n", "# use the hash built-in to create a fixed function like this:\n", "def fixed(o):\n", " try:\n", " hash(o)\n", " except TypeError:\n", " return False\n", " return True\n", "\n", "\n", "tf = (10, 'alpha', (1, 2)) # Contains no mutable items\n", "tm = (10, 'alpha', [1, 2]) # Contains a mutable item (list)\n", "fixed(tf)" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fixed(tm)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "## Unpacking sequences and iterables" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Unpacking is important because it avoids unnecessary and error-prone use of\n", "indexes to extract elements from sequences. \n", "- Also, unpacking works with any iterable object as the data source ——— including iterators, which don’t support index notation\n", "([])." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### parallel assignment" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "33.9425" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# the most visible form of unpacking is parallel assignment; that is, assigning items\n", "# from an iterable to a tuple of variables, as you can see in this example\n", "lax_coordinates = (33.9425, -118.408056)\n", "latitude, longitude = lax_coordinates # unpacking\n", "latitude" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "-118.408056" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "longitude" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### swapping values" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(-118.408056, 33.9425)" ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# An elegant application of unpacking is swapping the values of variables without using\n", "# a temporary variable\n", "latitude, longitude = longitude, latitude\n", "latitude, longitude" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Another example of unpacking is prefixing an argument with * when calling a function" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(2, 4)" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "divmod(20, 8)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(2, 4)" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t = (20, 8)\n", "divmod(*t)" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(2, 4)" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "quotient, remainder = divmod(*t)\n", "quotient, remainder" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "'id_rsa.pub'" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import os\n", "\n", "_, filename = os.path.split('/home/luciano/.ssh/id_rsa.pub')\n", "filename" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "### Using * to grab excess items" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Defining function parameters with *args to grab arbitrary excess arguments is a\n", "classic Python feature" ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(0, 1, [2, 3, 4])" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a, b, *rest = range(5) # parallel assignment and * grab\n", "a, b, rest" ] }, { "cell_type": "code", "execution_count": 42, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(0, 1, [2])" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a, b, *rest = range(3)\n", "a, b, rest" ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(0, 1, [])" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a, b, *rest = range(2)\n", "a, b, rest" ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(0, [1, 2], 3, 4)" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a, *body, c, d = range(5)\n", "a, body, c, d" ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "([0, 1], 2, 3, 4)" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "*head, b, c, d = range(5)\n", "head, b, c, d" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "### Unpacking with * in function calls and sequence literals" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(1, 2, 3, 4, (5, 6))" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def fun(a, b, c, d, *rest):\n", " return a, b, c, d, rest\n", "\n", "\n", "fun(*[1, 2], 3, *range(4, 7))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The * can also be used when defining list, tuple, or set literals" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "(0, 1, 2, 3, 4)" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "*range(4), 4" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "[0, 1, 2, 3, 4]" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[*range(4), 4]" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "{0, 1, 2, 3, 4, 5, 6, 7}" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "{*range(4), 4, *(5, 6, 7)}" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "### Nested unpacking\n", "#### Example 2-8. Unpacking nested tuples to access the longitude\n", "\n", "[02-array-seq/metro_lat_lon.py](02-array-seq/metro_lat_lon.py)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "## Pattern Matching with Sequences\n", "#### Example 2-9. Method from an imaginary Robot class" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [], "source": [ "# def handle_command(self, message):\n", "# match message:\n", "# case ['BEEPER', frequency, times]:\n", "# self.beep(times, frequency)\n", "# case ['NECK', angle]:\n", "# self.rotate_neck(angle)\n", "# case ['LED', ident, intensity]:\n", "# self.leds[ident].set_brightness(ident, intensity)\n", "# case ['LED', ident, red, green, blue]:\n", "# self.leds[ident].set_color(ident, red, green, blue)\n", "# case _:\n", "# raise InvalidCommand(message)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "#### Example 2-10. Destructuring nested tuples—requires Python ≥ 3.10.\n", "[02-array-seq/match_lat_lon.py](02-array-seq/match_lat_lon.py)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " | latitude | longitude\n", "Mexico City | 19.4333 | -99.1333\n", "New York-Newark | 40.8086 | -74.0204\n", "São Paulo | -23.5478 | -46.6358\n" ] } ], "source": [ "metro_areas = [\n", " ('Tokyo', 'JP', 36.933, (35.689722, 139.691667)),\n", " ('Delhi NCR', 'IN', 21.935, (28.613889, 77.208889)),\n", " ('Mexico City', 'MX', 20.142, (19.433333, -99.133333)),\n", " ('New York-Newark', 'US', 20.104, (40.808611, -74.020386)),\n", " ('São Paulo', 'BR', 19.649, (-23.547778, -46.635833)),\n", "]\n", "\n", "def main():\n", " print(f'{\"\":15} | {\"latitude\":>9} | {\"longitude\":>9}')\n", " for record in metro_areas:\n", " match record: # subject of this match is record —— i.e., each of the tuples in metro_areas\n", " # case clause has two parts: a pattern and a optional guard with the if keyword\n", " # this pattern matches a sequence with four items and the last item must be a two-item sequence\n", " case [name, _, _, (lat, lon)] if lon <= 0:\n", " print(f'{name:15} | {lat:9.4f} | {lon:9.4f}')\n", "main()" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "In general, a sequence pattern matches the subject if:\n", "1. The subject is a sequence and;\n", "2. The subject and the pattern have the same number of items and;\n", "3. Each corresponding item matches, including nested items." ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "### Pattern Matching Sequences in an Interpreter\n", "#### Example 2-11. Matching patterns without match/case.\n", "[02-array-seq/lispy/py3.9/lis.py](02-array-seq/lispy/py3.9/lis.py)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "#### Example 2-12. Pattern matching with match/case—requires Python ≥ 3.10.\n", "[02-array-seq/lispy/py3.10/lis.py](02-array-seq/lispy/py3.10/lis.py)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false }, "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Slicing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "list, str, tuple，and all sequence types in Python support slicing operations" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Why Slices and Range Exclude the Last Item" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The Pythonic convention of excluding the last item in slices and ranges works well\n", " with the zero-based indexing used in Python, C, and many other languages. Some\n", " convenient features of the convention are:\n", " - It’s easy to see the length of a slice or range when only the stop position is given:\n", " range(3) and my_list[:3] both produce three items.\n", " - It’s easy to compute the length of a slice or range when start and stop are given:\n", " just subtract stop - start.\n", " - It’s easy to split a sequence in two parts at any index x, without overlapping: sim\n", "ply get my_list[:x] and my_list[x:]. " ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[10, 20]" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l = [10, 20, 30, 40, 50, 60]\n", "\n", "l[:2] # split at 2" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[30, 40, 50, 60]" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l[2:]" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[10, 20, 30]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l[:3] # split at 3" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[40, 50, 60]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l[3:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Slice Objects" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'bye'" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s = 'bicycle'\n", "s[::3]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'elcycib'" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s[::-1] # stride is negative, returing items in reverse" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'eccb'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s[::-2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The notation a:b:c is only valid within [] when used as the indexing or subscript operator, and it produces a slice object: `slice(a, b, c)`\n", "- when evaluate the expression `seq[start:stop:step]`, Python calls `seq.__getitem__(slice(start, stop, step))`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-13. Line items from a flat-file invoice" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1909 Pimoroni PiBrella $17.50 3 $52.50\n", " $17.50 Pimoroni PiBrella \n", "1489 6mm Tactile Switch x20 $4.95 2 $9.90\n", " $4.95 6mm Tactile Switch x20 \n", "1510 Panavise Jr. - PV-201 $28.00 1 $28.00\n", " $28.00 Panavise Jr. - PV-201 \n", "1601 PiTFT Mini Kit 320x240 $34.95 1 $34.95\n", " $34.95 PiTFT Mini Kit 320x240 \n", "\n", " \n" ] } ], "source": [ "invoice = \"\"\"\n", "0.....6.................................40........52...55........\n", "1909 Pimoroni PiBrella $17.50 3 $52.50\n", "1489 6mm Tactile Switch x20 $4.95 2 $9.90\n", "1510 Panavise Jr. - PV-201 $28.00 1 $28.00\n", "1601 PiTFT Mini Kit 320x240 $34.95 1 $34.95\n", "\"\"\"\n", "\n", "SKU = slice(0, 5)\n", "DESCRIPTION = slice(5, 40)\n", "UNIT_PRICE = slice(40, 52)\n", "QUANTITY = slice(52, 55)\n", "ITEM_TOTAL = slice(55, None)\n", "\n", "line_items = invoice.split('\\n')[2:]\n", "\n", "for item in line_items:\n", " print(item)\n", " print(item[UNIT_PRICE], item[DESCRIPTION]) # 用slice object表示序列的切片操作" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Multidimensional Slicing and Ellipsis(...)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Slicing includes additional features such as multidimensional slices and ellipsis(...) notation.\n", "- The [] operator can also take multiple indexes or slices separated by commas" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- numpy.ndarray:\n", " - `a[i, j]`, two indexes\n", " - `a[m:n, k:l]`, two-dimensional slice " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Ellipsis\n", "- The ellipsis—written with three full stops (...) and not … (Unicode U+2026)—is recognized as a token by the Python parser. It is an alias to the Ellipsis object, the single instance of the ellipsis class.\n", "- passed ... as an argument to functions:\n", " - `f(a, ..., z)`\n", "- as part of a slice:\n", " - `a[i:...]`\n", "- Numpy uses ... as a shorcut when slicing multi-dimensional arrays\n", " - x is four-dimensional array, `x[i, ...]` is a shortcut for `x[i, :, :, :,]`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Assigning to Slices" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- above-ammentioned slices is used to extract information from sequences, but Slices can also be used to change mutable sequences\n", "- Mutable sequences can be grafted, excised, and otherwise modified in place using slice notation on the lefthand side of an assignment statement or as the target of a del statement." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l = list(range(10))\n", "l" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, 1, 20, 30, 5, 6, 7, 8, 9]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l[2:5] = [20, 30]\n", "l" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, 1, 20, 30, 5, 8, 9]" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "del l[5:7]\n", "l" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, 1, 20, 11, 5, 22, 9]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l[3::2] = [11, 22]\n", "l" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By design, this example raises an exception::" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TypeError('can only assign an iterable')\n" ] } ], "source": [ "try:\n", " l[2:5] = 100\n", "except TypeError as e:\n", " print(repr(e))" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, 1, 100]" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l[2:5] = [100]\n", "l" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using + and * with Sequences" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Both + and * always create a new object, and never change their operands" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3, 1, 2, 3]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l = [1, 2, 3]\n", "l * 5 # concatenate multiple copies of the same sequence with * operators, new sequence of the same type is created but l is not modified" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1, 2, 3]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l # l was not modified" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'abcdabcdabcdabcdabcd'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "5 * 'abcd'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Building Lists of Lists" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-14. A list with three lists of length 3 can represent a tic-tac-toe board" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['_', '_', '_'], ['_', '_', '_'], ['_', '_', '_']]" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "board = [['_'] * 3 for i in range(3)]\n", "board" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['_', '_', '_'], ['_', '_', 'X'], ['_', '_', '_']]" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "board[1][2] = 'X'\n", "board" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-15. A list with three references to the same list is useless" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['_', '_', '_'], ['_', '_', '_'], ['_', '_', '_']]" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "weird_board = [['_'] * 3] * 3 # The outer list is made of three references to the same inner list\n", "weird_board" ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['_', '_', 'O'], ['_', '_', 'O'], ['_', '_', 'O']]" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "weird_board[1][2] = 'O'\n", "weird_board" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- the above result reveals that all rows of weird_board are aliases referring to the same object" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Explanation" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['_', '_', '_'], ['_', '_', '_'], ['_', '_', '_']]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# list comprehension from example 2-14 is equivalent to this code\n", "board = []\n", "for i in range(3):\n", " row = ['_'] * 3 # each iteration builds a new row and append it to board\n", " board.append(row)\n", "board" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['_', '_', '_'], ['_', '_', '_'], ['X', '_', '_']]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "board[2][0] = 'X'\n", "board" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# example 2-15 is same as this code\n", "row = ['_'] * 3\n", "board = []\n", "for i in range(3):\n", " board.append(row) # the same row is appended three times to board" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[['x', '_', '_'], ['x', '_', '_'], ['x', '_', '_']]" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "board[2][0] = 'x'\n", "board" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Augmented Assignment with Sequences" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The augmented assignment operators += and *= behave quite differently, depending on the first operand.\n", "- The special method that makes += work is \\_\\_iadd\\_\\_ (for “in-place addition”).\n", "- However, if \\_\\_iadd\\_\\_ is not implemented, Python falls back to calling \\_\\_add\\_\\_. \n", "- Operator *=, which is implemented via \\_\\_imul\\_\\_" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "l = [1, 2, 3]\n", "idl = id(l)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2316354397824" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# NBVAL_IGNORE_OUTPUT\n", "idl" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[1, 2, 3, 1, 2, 3]" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l *= 2\n", "l" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "id(l) == idl # same list" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "t = (1, 2, 3) # t is immutable sequeces\n", "idt = id(t)" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2316354442816" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# NBVAL_IGNORE_OUTPUT\n", "idt" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t *= 2 # preduce new tuple and t reference it\n", "id(t) == idt # new tuple" ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(1, 2, 3, 1, 2, 3)" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Repeated concatenation of immutable sequences is inefficient, because instead of just appending new items, the interpreter has to copy the whole target sequence to create a new one with the new items concatenated.\n", "- str is an exception to this description. Because string building with += in loops is so common in real codeba\n", "ses, CPython is optimized for this use case. Instances of str are allocated in memory with extra room, so that\n", " concatenation does not require copying the whole string every time" ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "s = 'abc'\n", "idt = id(s)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "140710833721696" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "idt" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "False" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s *= 2\n", "idt == id(s)" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'abcabc'" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### A += Assignment Puzzler\n", "#### Example 2-16. A riddle" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TypeError(\"'tuple' object does not support item assignment\")\n" ] } ], "source": [ "t = (1, 2, [30, 40])\n", "try:\n", " t[2] += [50, 60]\n", "except TypeError as e:\n", " print(repr(e))" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "#### Example 2-17. The unexpected result: item t2 is changed and an exception is raised" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(1, 2, [30, 40, 50, 60])" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The result of example 2-16 is both raise exception and t[2] was be changed\n", "- test by https://pythontutor.com/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Lessons from example 2-16\n", "- Avoid putting mutable items in tuples.\n", "- Augmented assignment is not an atomic operation—we just saw it throwing an exception after doing part of its job.\n", "- Inspecting Python bytecode is not too difficult, and can be helpful to see what is going on under the hood." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-18. Bytecode for the expression s[a] += b" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " 0 0 RESUME 0\n", "\n", " 1 2 LOAD_NAME 0 (s)\n", " 4 LOAD_NAME 1 (a)\n", " 6 COPY 2\n", " 8 COPY 2\n", " 10 BINARY_SUBSCR\n", " 20 LOAD_NAME 2 (b)\n", " 22 BINARY_OP 13 (+=)\n", " 26 SWAP 3\n", " 28 SWAP 2\n", " 30 STORE_SUBSCR\n", " 34 LOAD_CONST 0 (None)\n", " 36 RETURN_VALUE\n" ] } ], "source": [ "import dis\n", "\n", "dis.dis('s[a] += b')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## list.sort and the sorted Built-In Function" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- The list.sort method sorts a list in place—that is, without making a copy. It returns None to remind us that it changes the receiver(Receiver is the target of a method call, the object bound to self in the method body) and does not create a new list.\n", "- This is an important Python API convention: **functions or methods that change an object in place should return None to make it clear to the caller that the receiver was changed, and no new object was created**.\n", "- The convention of returning None to signal in-place changes has a drawback: we cannot cascade calls to those methods. In contrast, methods that return new objects (e.g., all str methods) can be cascaded in the fluent interface style.\n", "- In contrast, the built-in function sorted creates a new list and returns it. It accepts any iterable object as an argument, including immutable sequences and generators. Regardless of the type of iterable given to sorted, it always returns a\n", " newly created list." ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2316354507072" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fruits = ['grape', 'raspberry', 'apple', 'banana']\n", "idt = id(fruits)\n", "idt" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['apple', 'banana', 'grape', 'raspberry']" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted(fruits) # This produces a new list of strings sorted alphabetically" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['grape', 'raspberry', 'apple', 'banana']" ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fruits # not changed" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['raspberry', 'grape', 'banana', 'apple']" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# reverse is true the items are returned in descending order (i.e., by reversing the comparison of the items). The default is False\n", "sorted(fruits, reverse=True) " ] }, { "cell_type": "code", "execution_count": 53, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['apple', 'grape', 'banana', 'raspberry']" ] }, "execution_count": 53, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# key is a one-argument function that will be applied to each item to produce its sorting key\n", "# key=len, will sort the strings by character length\n", "# key=min() or max() with sort by min or max of each item\n", "# key=str.lower for a list of strings will perform a case-insensitive sort\n", "# thee default sort key is the identity function(i.e., the items themselves are compared)\n", "sorted(fruits, key=len)" ] }, { "cell_type": "code", "execution_count": 54, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['raspberry', 'banana', 'apple', 'grape']" ] }, "execution_count": 54, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted(fruits, key=len, reverse=True)" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['grape', 'raspberry', 'apple', 'banana']" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fruits" ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['apple', 'banana', 'grape', 'raspberry']" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "fruits.sort() # in-place sort and return None\n", "fruits" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2316354507072" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" } ], "source": [ "id(fruits)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- bisect module\n", "- https://www.fluentpython.com/\n", "- https://www.fluentpython.com/extra/ordered-sequences-with-bisect/" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## When a List Is Not the Answer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### better than list\n", "- An array saves a lot of memory when you need to handle millions of floating-point values\n", "- On the other hand, if you are constantly adding and removing items from opposite ends of a list, it’s good to know that a\n", " deque (double-ended queue) is a more efficient FIFO14 data structure.\n", "- If your code frequently checks whether an item is present in a collection (e.g., item in my_collection), consider using a set for my_collection, especially if it holds a large number of items. Sets are optimized for fast membership checking. They are also iterable, but they are not sequences because the ordering of set items is unspecified. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Arrays" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- If a list only contains numbers, an array.array is a more efficient replacement.\n", " Arrays support all mutable sequence operations (including .pop, .insert,\n", " and .extend), as well as additional methods for fast loading and saving, such\n", " as .frombytes and .tofile\n", " - When creating an array, you provide a typecode, a letter to determine the underlying C\n", " type used to store each item in the array. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-19. Creating, saving, and loading a large array of floats" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8190492979077034" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from array import array\n", "from random import random, seed\n", "seed(10) # Use seed to make the output consistent\n", "\n", "# Create an array of double-precision floats (typecode 'd') from any iterable object —in this case, a generator expression.\n", "floats = array('d', (random() for i in range(10 ** 7)))\n", "floats[-1]" ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [], "source": [ "# save the array to binary file\n", "with open('floats.bin', 'wb') as fp:\n", " floats.tofile(fp)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.8190492979077034" ] }, "execution_count": 79, "metadata": {}, "output_type": "execute_result" } ], "source": [ "floats2 = array('d') # Create an empty array of doubles\n", "\n", "with open('floats.bin', 'rb') as fp:\n", " floats2.fromfile(fp, 10 ** 7)\n", "\n", "floats2[-1]" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 80, "metadata": {}, "output_type": "execute_result" } ], "source": [ "floats2 == floats" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Memory Views" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "#### Example 2-20. Handling 6 bytes memory of as 1×6, 2×3, and 3×2 views" ] }, { "cell_type": "code", "execution_count": 81, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "[0, 1, 2, 3, 4, 5]" ] }, "execution_count": 81, "metadata": {}, "output_type": "execute_result" } ], "source": [ "octets = array('B', range(6))\n", "m1 = memoryview(octets)\n", "m1.tolist()" ] }, { "cell_type": "code", "execution_count": 82, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "[[0, 1, 2], [3, 4, 5]]" ] }, "execution_count": 82, "metadata": {}, "output_type": "execute_result" } ], "source": [ "m2 = m1.cast('B', [2, 3])\n", "m2.tolist()" ] }, { "cell_type": "code", "execution_count": 83, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "[[0, 1], [2, 3], [4, 5]]" ] }, "execution_count": 83, "metadata": {}, "output_type": "execute_result" } ], "source": [ "m3 = m1.cast('B', [3, 2])\n", "m3.tolist()" ] }, { "cell_type": "code", "execution_count": 84, "metadata": { "collapsed": false, "pycharm": { "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "array('B', [0, 1, 2, 33, 22, 5])" ] }, "execution_count": 84, "metadata": {}, "output_type": "execute_result" } ], "source": [ "m2[1,1] = 22\n", "m3[1,1] = 33\n", "octets" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-21. Changing the value of an 16-bit integer array item by poking one of its bytes" ] }, { "cell_type": "code", "execution_count": 85, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5" ] }, "execution_count": 85, "metadata": {}, "output_type": "execute_result" } ], "source": [ "numbers = array('h', [-2, -1, 0, 1, 2])\n", "memv = memoryview(numbers)\n", "len(memv)" ] }, { "cell_type": "code", "execution_count": 86, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "-2" ] }, "execution_count": 86, "metadata": {}, "output_type": "execute_result" } ], "source": [ "memv[0]" ] }, { "cell_type": "code", "execution_count": 87, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[254, 255, 255, 255, 0, 0, 1, 0, 2, 0]" ] }, "execution_count": 87, "metadata": {}, "output_type": "execute_result" } ], "source": [ "memv_oct = memv.cast('B')\n", "memv_oct.tolist()" ] }, { "cell_type": "code", "execution_count": 88, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array('h', [-2, -1, 1024, 1, 2])" ] }, "execution_count": 88, "metadata": {}, "output_type": "execute_result" } ], "source": [ "memv_oct[5] = 4\n", "numbers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### NumPy" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-22. Basic operations with rows and columns in a numpy.ndarray" ] }, { "cell_type": "code", "execution_count": 89, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])" ] }, "execution_count": 89, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "\n", "a = np.arange(12)\n", "a" ] }, { "cell_type": "code", "execution_count": 90, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "numpy.ndarray" ] }, "execution_count": 90, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(a)" ] }, { "cell_type": "code", "execution_count": 91, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(12,)" ] }, "execution_count": 91, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.shape" ] }, { "cell_type": "code", "execution_count": 92, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 0, 1, 2, 3],\n", " [ 4, 5, 6, 7],\n", " [ 8, 9, 10, 11]])" ] }, "execution_count": 92, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.shape = 3, 4\n", "a" ] }, { "cell_type": "code", "execution_count": 93, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 8, 9, 10, 11])" ] }, "execution_count": 93, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[2]" ] }, { "cell_type": "code", "execution_count": 94, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "9" ] }, "execution_count": 94, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[2, 1]" ] }, { "cell_type": "code", "execution_count": 95, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1, 5, 9])" ] }, "execution_count": 95, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a[:, 1]" ] }, { "cell_type": "code", "execution_count": 96, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[ 0, 4, 8],\n", " [ 1, 5, 9],\n", " [ 2, 6, 10],\n", " [ 3, 7, 11]])" ] }, "execution_count": 96, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a.transpose()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-22. Loading, saving, and vectorized operations" ] }, { "cell_type": "code", "execution_count": 97, "metadata": {}, "outputs": [], "source": [ "with open('floats-1M-lines.txt', 'wt') as fp:\n", " for _ in range(1_000_000):\n", " fp.write(f'{random()}\\n')" ] }, { "cell_type": "code", "execution_count": 98, "metadata": {}, "outputs": [], "source": [ "floats = np.loadtxt('floats-1M-lines.txt')" ] }, { "cell_type": "code", "execution_count": 99, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.06078257, 0.61741189, 0.84349987])" ] }, "execution_count": 99, "metadata": {}, "output_type": "execute_result" } ], "source": [ "floats[-3:]" ] }, { "cell_type": "code", "execution_count": 100, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0.03039128, 0.30870594, 0.42174994])" ] }, "execution_count": 100, "metadata": {}, "output_type": "execute_result" } ], "source": [ "floats *= .5\n", "floats[-3:]" ] }, { "cell_type": "code", "execution_count": 101, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 101, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from time import perf_counter as pc\n", "\n", "t0 = pc()\n", "floats /= 3\n", "(pc() - t0) < 0.01" ] }, { "cell_type": "code", "execution_count": 102, "metadata": {}, "outputs": [], "source": [ "np.save('floats-1M', floats)\n", "floats2 = np.load('floats-1M.npy', 'r+')\n", "floats2 *= 6" ] }, { "cell_type": "code", "execution_count": 103, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "memmap([0.06078257, 0.61741189, 0.84349987])" ] }, "execution_count": 103, "metadata": {}, "output_type": "execute_result" } ], "source": [ "floats2[-3:]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Deques and Other Queues" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example 2-23. Working with a deque" ] }, { "cell_type": "code", "execution_count": 104, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "deque([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])" ] }, "execution_count": 104, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import collections\n", "\n", "dq = collections.deque(range(10), maxlen=10)\n", "dq" ] }, { "cell_type": "code", "execution_count": 105, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "deque([7, 8, 9, 0, 1, 2, 3, 4, 5, 6])" ] }, "execution_count": 105, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dq.rotate(3)\n", "dq" ] }, { "cell_type": "code", "execution_count": 106, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "deque([1, 2, 3, 4, 5, 6, 7, 8, 9, 0])" ] }, "execution_count": 106, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dq.rotate(-4)\n", "dq" ] }, { "cell_type": "code", "execution_count": 107, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "deque([-1, 1, 2, 3, 4, 5, 6, 7, 8, 9])" ] }, "execution_count": 107, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dq.appendleft(-1)\n", "dq" ] }, { "cell_type": "code", "execution_count": 108, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "deque([3, 4, 5, 6, 7, 8, 9, 11, 22, 33])" ] }, "execution_count": 108, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dq.extend([11, 22, 33])\n", "dq" ] }, { "cell_type": "code", "execution_count": 109, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "deque([40, 30, 20, 10, 3, 4, 5, 6, 7, 8])" ] }, "execution_count": 109, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dq.extendleft([10, 20, 30, 40])\n", "dq" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Soapbox" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Mixed bag lists" ] }, { "cell_type": "code", "execution_count": 110, "metadata": {}, "outputs": [], "source": [ "l = [28, 14, '28', 5, '9', '1', 0, 6, '23', 19]" ] }, { "cell_type": "code", "execution_count": 111, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TypeError(\"'<' not supported between instances of 'str' and 'int'\")\n" ] } ], "source": [ "try:\n", " sorted(l)\n", "except TypeError as e:\n", " print(repr(e))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Key is Brilliant" ] }, { "cell_type": "code", "execution_count": 112, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, '1', 5, 6, '9', 14, 19, '23', 28, '28']" ] }, "execution_count": 112, "metadata": {}, "output_type": "execute_result" } ], "source": [ "l = [28, 14, '28', 5, '9', '1', 0, 6, '23', 19]\n", "\n", "sorted(l, key=int)" ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[0, '1', 14, 19, '23', 28, '28', 5, 6, '9']" ] }, "execution_count": 113, "metadata": {}, "output_type": "execute_result" } ], "source": [ "sorted(l, key=str)" ] }, { "cell_type": "code", "execution_count": 113, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 2 }