{ "cells": [ { "cell_type": "markdown", "metadata": { "nbsphinx": "hidden" }, "source": [ "
\n", " `21 Microservices Architecture Patterns Study
\n", "\n", "

Chapter 2. Repository Pattern

\n", "\n", "
\n", " Joseph Kim <cloudeyes@gmail.com>
\n", " Dec 23. 2020
\n", "
\n", "\n", "
\n", " \n", "
\n", "\n", "
\n", " Download Jupyter Notebook\n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 02. Repository Pattern\n", "\n", "## Prerequisites" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The folder structure" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[01;34m.\u001b[00m\n", "├── 02-repository-pattern.ipynb\n", "├── \u001b[01;34mapp\u001b[00m\n", "│   ├── \u001b[01;34mdomain\u001b[00m\n", "│   │   └── models.py\n", "│   ├── __init__.py\n", "│   └── services.py\n", "├── requirements.txt\n", "└── \u001b[01;34mtests\u001b[00m\n", " ├── __init__.py\n", " └── test_models.py\n", "\n", "3 directories, 7 files\n" ] } ], "source": [ "!tree -I '__pycache__|build|images|plantuml*|exports|examples'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Install requirements " ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "sqlalchemy\n", "pytest\n", "git+https://github.com/AllanDaemon/mypy.git@0edf1233672117c4555759c5a91461a502ddce5d\n" ] } ], "source": [ "!cat ./requirements.txt" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "nbsphinx": "hidden", "tags": [ "hidden-output" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: sqlalchemy in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from -r ./requirements.txt (line 1)) (1.3.22)\n", "Requirement already satisfied: pytest in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from -r ./requirements.txt (line 2)) (6.1.2)\n", "Collecting git+https://github.com/AllanDaemon/mypy.git@0edf1233672117c4555759c5a91461a502ddce5d (from -r ./requirements.txt (line 3))\n", " Cloning https://github.com/AllanDaemon/mypy.git (to revision 0edf1233672117c4555759c5a91461a502ddce5d) to /tmp/pip-req-build-g9qozp5y\n", " Installing build dependencies ... \u001b[?25ldone\n", "\u001b[?25h Getting requirements to build wheel ... \u001b[?25ldone\n", "\u001b[?25h Preparing wheel metadata ... \u001b[?25ldone\n", "\u001b[?25hRequirement already satisfied: typing-extensions>=3.7.4 in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from mypy==0.800+dev.0edf1233672117c4555759c5a91461a502ddce5d->-r ./requirements.txt (line 3)) (3.7.4.3)\n", "Requirement already satisfied: mypy-extensions<0.5.0,>=0.4.3 in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from mypy==0.800+dev.0edf1233672117c4555759c5a91461a502ddce5d->-r ./requirements.txt (line 3)) (0.4.3)\n", "Requirement already satisfied: typed-ast<1.5.0,>=1.4.0 in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from mypy==0.800+dev.0edf1233672117c4555759c5a91461a502ddce5d->-r ./requirements.txt (line 3)) (1.4.1)\n", "Requirement already satisfied: attrs>=17.4.0 in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from pytest->-r ./requirements.txt (line 2)) (20.3.0)\n", "Requirement already satisfied: iniconfig in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from pytest->-r ./requirements.txt (line 2)) (1.1.1)\n", "Requirement already satisfied: packaging in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from pytest->-r ./requirements.txt (line 2)) (20.8)\n", "Requirement already satisfied: pluggy<1.0,>=0.12 in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from pytest->-r ./requirements.txt (line 2)) (0.13.1)\n", "Requirement already satisfied: py>=1.8.2 in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from pytest->-r ./requirements.txt (line 2)) (1.10.0)\n", "Requirement already satisfied: toml in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from pytest->-r ./requirements.txt (line 2)) (0.10.2)\n", "Requirement already satisfied: pyparsing>=2.0.2 in /home/ykkim/conda/envs/lab/lib/python3.9/site-packages (from packaging->pytest->-r ./requirements.txt (line 2)) (2.4.7)\n" ] } ], "source": [ "!pip install -r ./requirements.txt" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Tests" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m============================= test session starts ==============================\u001b[0m\n", "platform linux -- Python 3.9.1, pytest-6.1.2, py-1.10.0, pluggy-0.13.1\n", "rootdir: /home/ykkim/notebooks/2021-msa-study/02-repository-pattern\n", "plugins: flask-1.1.0, anyio-2.0.2\n", "collected 12 items \u001b[0m\n", "\n", "tests/test_models.py \u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m.\u001b[0m\u001b[32m [100%]\u001b[0m\n", "\n", "\u001b[32m============================== \u001b[32m\u001b[1m12 passed\u001b[0m\u001b[32m in 0.04s\u001b[0m\u001b[32m ==============================\u001b[0m\n" ] } ], "source": [ "!pytest" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "app/services.py:3: \u001b[1m\u001b[31merror:\u001b[m Cannot find implementation or library stub for module named 'app.domain.models'\u001b[m\n", "app/services.py:3: \u001b[34mnote:\u001b[m See \u001b[4mhttps://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports\u001b[m\n", "app/services.py:10: \u001b[1m\u001b[31merror:\u001b[m Returning Any from function declared to return \u001b[m\u001b[1m\"str\"\u001b[m\u001b[m\n", "\u001b[1m\u001b[31mFound 2 errors in 1 file (checked 2 source files)\u001b[m\n" ] } ], "source": [ "!mypy --strict -p app" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Persisting Our Domain Model" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Reminder: Our Model" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "tags": [ "hidden-input" ] }, "outputs": [ { "data": { "image/svg+xml": [ "PurchasingDeptPurchasingDept. UserOrderid: str«valueobject»OrderLineorderid: strsku: strqty: intBatchreference: strsku: streta: date_purchased_quantity: int_allocations: [OrderLine]Customerplacescomprisesmultiplepurchases" ], "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%plantuml\n", "\n", "@startuml\n", "\n", "allow_mixing\n", "left to right direction\n", "\n", "class Order {\n", " * id: str\n", "}\n", "\n", "class OrderLine << valueobject >> {\n", " * orderid: str\n", " * sku: str\n", " * qty: int\n", "}\n", "\n", "class Batch {\n", " * reference: str\n", " sku: str\n", " eta: date\n", " --\n", " _purchased_quantity: int\n", " _allocations: [OrderLine]\n", "}\n", "\n", "actor Customer\n", "actor \"Purchasing\\nDept. User\"\n", "\n", "Customer --> Order : places\n", "Order::id o--> OrderLine::orderid : \"comprises\\nmultiple\"\n", "Batch::_allocations o--> OrderLine\n", "\"Purchasing\\nDept. User\" --> Batch : \"purchases\"\n", " \n", "@enduml" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Customers place *orders* for *products*.\n", "\n", "- **Product**\n", " - identified by *SKU*(Stock Keeping Unit)\n", "- **Order**\n", " - identified by an *order reference* \n", " - comprises mutliple *order lines*\n", "- **OrderLine**\n", " - has a *SKU* and a *quantity*\n", " \n", "#### The purchasing department orders small *batches* of stock\n", "\n", "- **Batch**\n", " - has a unique ID(*reference*), a *SKU*, and a *quantity* \n", " - has an ETA if they are currently shipping\n", " - or they may be in *warehouse stock*.\n", " \n", "#### We need to **allocate** *order lines* to *batches*.\n", "\n", "- When we allocate x units of stock to a batch, the *available quantity* is reduced by x." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The \"Normal\" ORM Way: Model Depends on ORM" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "jupyter": { "source_hidden": true }, "tags": [ "hidden-input" ] }, "outputs": [ { "data": { "image/svg+xml": [ "Orderid: IntegerBatchid: Integerreference: String(255)sku: String(255)qty: IntegerAllocationbatch_idorderline_idOrderLineid: Integerorder_id: Integersku: String(255)qty: Integereta: Date" ], "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%plantuml\n", "\n", "@startuml\n", "\n", "left to right direction\n", "\n", "entity Order {\n", " * **id**: Integer\n", "}\n", "\n", "entity Batch {\n", " * **id**: Integer\n", " --\n", " * **reference**: String(255)\n", " * sku: String(255)\n", " * qty: Integer\n", "}\n", "\n", "entity Allocation {\n", " * **batch_id**\n", " * **orderline_id**\n", "}\n", "\n", "entity OrderLine {\n", " * **id**: Integer\n", " --\n", " * **order_id: Integer**\n", " * sku: String(255)\n", " * qty: Integer\n", " eta: Date\n", "}\n", "\n", "\n", "Order::id ||..|{ OrderLine::order_id\n", "OrderLine::id ||..|{ Allocation::orderline_id \n", "Batch::id ||..|{ Allocation::batch_id \n", "\n", "\n", "@enduml" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "from sqlalchemy import Column, ForeignKey, Integer, String, Date, Table\n", "from sqlalchemy.ext.declarative import declarative_base\n", "from sqlalchemy.orm import relationship\n", "\n", "Base = declarative_base()\n", "\n", "class _Order(Base):\n", " __tablename__ = 'order'\n", " id = Column(Integer, primary_key=True, autoincrement=True)\n", " orderlines = relationship('_OrderLine', collection_class=set)\n", " \n", "class _OrderLine(Base):\n", " __tablename__ = 'order_line'\n", " id = Column(Integer, primary_key=True, autoincrement=True)\n", " sku = Column(String(255))\n", " qty = Column(Integer)\n", " orderid = Column(Integer, ForeignKey('order.id'))\n", " order = relationship(_Order)\n", "\n", "allocation = Table('allocation', Base.metadata,\n", " Column('orderline_id', Integer, ForeignKey('order_line.id'), primary_key=True),\n", " Column('batch_id', Integer, ForeignKey('batch.id'), primary_key=True)\n", ")\n", " \n", "class _Batch(Base):\n", " __tablename__ = 'batch'\n", " id = Column(Integer, primary_key=True, autoincrement=True)\n", " reference = Column(String(255), unique=True)\n", " sku = Column(String(255))\n", " eta = Column(Date)\n", " _purchased_quantity = Column(Integer)\n", " _allocations = relationship(_OrderLine, secondary=allocation, collection_class=set)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CREATE TABLE \"order\" (\n", "\tid INTEGER NOT NULL, \n", "\tPRIMARY KEY (id)\n", ")\n", "\n", "CREATE TABLE batch (\n", "\tid INTEGER NOT NULL, \n", "\treference VARCHAR(255), \n", "\tsku VARCHAR(255), \n", "\teta DATE, \n", "\t_purchased_quantity INTEGER, \n", "\tPRIMARY KEY (id), \n", "\tUNIQUE (reference)\n", ")\n", "\n", "CREATE TABLE order_line (\n", "\tid INTEGER NOT NULL, \n", "\tsku VARCHAR(255), \n", "\tqty INTEGER, \n", "\torderid INTEGER, \n", "\tPRIMARY KEY (id), \n", "\tFOREIGN KEY(orderid) REFERENCES \"order\" (id)\n", ")\n", "\n", "CREATE TABLE allocation (\n", "\torderline_id INTEGER NOT NULL, \n", "\tbatch_id INTEGER NOT NULL, \n", "\tPRIMARY KEY (orderline_id, batch_id), \n", "\tFOREIGN KEY(orderline_id) REFERENCES order_line (id), \n", "\tFOREIGN KEY(batch_id) REFERENCES batch (id)\n", ")\n", "\n", "\n" ] } ], "source": [ "def init_engine(url, metadata):\n", " import io, sys, re, logging\n", " from sqlalchemy import create_engine\n", " logger = logging.getLogger(\"sqlalchemy.engine.base.Engine\")\n", " out = io.StringIO()\n", " logger.addHandler(logging.StreamHandler(out))\n", " \n", " engine = create_engine(url, echo=True) # temporary memory db\n", " metadata.create_all(engine)\n", " \n", " print(''.join(re.findall('CREATE TABLE.*?\\n\\n', out.getvalue(), re.DOTALL)))\n", " return engine\n", " \n", "engine = init_engine('sqlite://', Base.metadata)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Inverting the Dependency: ORM Depends on Model\n", "\n", "Using SQLAlchemy's \"classical mapping\"." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "from sqlalchemy import MetaData, Table, Column, ForeignKey, Integer, String, Date\n", "from sqlalchemy.orm import mapper, relationship\n", "from app.domain.models import Batch, OrderLine\n", "\n", "metadata = MetaData()\n", "\n", "order_line = Table(\n", " 'order_line', metadata,\n", " Column('id', Integer, primary_key=True, autoincrement=True),\n", " Column('sku', String(255)),\n", " Column('qty', Integer, nullable=False),\n", " Column('orderid', String(255)),\n", " extend_existing=True\n", ")\n", "\n", "allocation = Table('allocation', metadata,\n", " Column('orderline_id', Integer, ForeignKey('order_line.id'), primary_key=True),\n", " Column('batch_id', Integer, ForeignKey('batch.id'), primary_key=True),\n", " extend_existing=True\n", ")\n", "\n", "batch = Table(\n", " 'batch', metadata,\n", " Column('id', Integer, primary_key=True, autoincrement=True),\n", " Column('reference', String(255), unique=True),\n", " Column('_purchased_quantity', Integer),\n", " Column('sku', String(255)),\n", " Column('eta', Date, nullable=True),\n", " extend_existing=True\n", ")\n", "\n", "batch_mapper = mapper(Batch, batch, properties={\n", " '_allocations': relationship(OrderLine, secondary=allocation, \n", " collection_class=set, lazy='joined'),\n", "},)\n", "\n", "order_line_mapper = mapper(OrderLine, order_line)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CREATE TABLE order_line (\n", "\tid INTEGER NOT NULL, \n", "\tsku VARCHAR(255), \n", "\tqty INTEGER NOT NULL, \n", "\torderid VARCHAR(255), \n", "\tPRIMARY KEY (id)\n", ")\n", "\n", "CREATE TABLE batch (\n", "\tid INTEGER NOT NULL, \n", "\treference VARCHAR(255), \n", "\t_purchased_quantity INTEGER, \n", "\tsku VARCHAR(255), \n", "\teta DATE, \n", "\tPRIMARY KEY (id), \n", "\tUNIQUE (reference)\n", ")\n", "\n", "CREATE TABLE allocation (\n", "\torderline_id INTEGER NOT NULL, \n", "\tbatch_id INTEGER NOT NULL, \n", "\tPRIMARY KEY (orderline_id, batch_id), \n", "\tFOREIGN KEY(orderline_id) REFERENCES order_line (id), \n", "\tFOREIGN KEY(batch_id) REFERENCES batch (id)\n", ")\n", "\n", "\n" ] } ], "source": [ "from contextlib import contextmanager, AbstractContextManager\n", "\n", "from sqlalchemy import create_engine\n", "from sqlalchemy.orm import sessionmaker\n", "from sqlalchemy.orm.session import Session\n", "\n", "engine = init_engine('sqlite://', metadata) # temporary memory db\n", "\n", "SqliteSessionMaker = sessionmaker(engine, expire_on_commit=False)\n", "SessionFactory = AbstractContextManager[Session]\n", "\n", "@contextmanager\n", "def session() -> SessionFactory:\n", " '''`with session` 블록을 이용한 자동 리소스 반환을 구현합니다.'''\n", " session = SqliteSessionMaker()\n", " try:\n", " yield session\n", " finally:\n", " session.close()" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ \u001b[95mtest_orderline_mapper_can_load_lines\u001b[0m\n" ] } ], "source": [ "from app import mytest\n", "\n", "mytest.init(__name__)\n", "\n", "@mytest.unit\n", "def test_orderline_mapper_can_load_lines(session):\n", " with session as db:\n", " db.execute('''\n", " INSERT INTO order_line (orderid, sku, qty) VALUES\n", " ('order1', 'RED-CHAIR', 12),\n", " ('order2', 'RED-TABLE', 13),\n", " ('order3', 'BLUE-LIPSTICK', 14)\n", " ''')\n", " expected = [\n", " OrderLine('order1', 'RED-CHAIR', 12),\n", " OrderLine('order2', 'RED-TABLE', 13),\n", " OrderLine('order3', 'BLUE-LIPSTICK', 14),\n", " ]\n", " assert expected == db.query(OrderLine).all()" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ \u001b[95mtest_orderline_mapper_can_save_lines\u001b[0m\n" ] } ], "source": [ "@mytest.unit\n", "def test_orderline_mapper_can_save_lines(session):\n", " with session as db:\n", " new_line = OrderLine('order1', 'DECORATIVE-WIDGET', 12)\n", " db.add(new_line)\n", " db.flush()\n", " \n", " rows = list(db.execute('SELECT orderid, sku, qty FROM order_line'))\n", " assert [('order1', 'DECORATIVE-WIDGET', 12)] == rows" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Introducing the Repository Pattern\n", "\n", "- An abstraction over persistent storage. \n", "- Hides details of data access by *pretending that all of data is in memory*." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "jupyter": { "source_hidden": true }, "tags": [ "hidden-input" ] }, "outputs": [ { "data": { "image/svg+xml": [ "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Application Layer\n", " Repository\n", " Database Layer\n", " Domain Model\n", " Objects\n", " \n", "" ], "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%ditaa repository-pattern\n", "+----------------------------------------+\n", "| Application Layer |\n", "+----------------------------------------+\n", " |^ /----------------\\\n", " ||------| Domain Model |\n", " || | Objects |\n", " V| \\----------------/\n", "+----------------------------------------+\n", "| Repository |\n", "+----------------------------------------+\n", " |\n", " V\n", "+----------------------------------------+\n", "| Database Layer |\n", "+----------------------------------------+ " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The Repository in the Abstract" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "from typing import Protocol, Iterable, Set\n", "from contextlib import ContextDecorator\n", "import abc\n", "\n", "class AbstractRepository(abc.ABC, ContextDecorator):\n", " \n", " def __enter__(self):\n", " return self\n", " \n", " def __exit__(self):\n", " self.close()\n", " return False \n", " \n", " @abc.abstractmethod\n", " def close(self, batch: Batch) -> None:\n", " pass\n", " \n", " @abc.abstractmethod\n", " def add(self, batch: Batch):\n", " raise NotImplementedError\n", "\n", " @abc.abstractmethod\n", " def get(self, reference) -> Batch:\n", " raise NotImplementedError\n", " \n", " @abc.abstractmethod\n", " def list(self) -> list[Batch]:\n", " raise NotImplementedError \n", " \n", " @abc.abstractmethod\n", " def clear(self) -> None:\n", " raise NotImplementedError\n", "\n", "# Alternative approache: using `Protocol`\n", "class RepositoryProtocol(Protocol):\n", " \n", " def add(self, batch: Batch) -> None:\n", " pass\n", "\n", " def get(self, reference) -> Batch:\n", " pass\n", " \n", " def list(self) -> list[Batch]:\n", " pass\n", " \n", " def clear(self) -> None:\n", " pass" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### What is the Trade-Off?\n", "\n", "Introducing an extra layer of abstraction, \n", "- Will reduce complexity overall\n", "- But add complexity locally" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Concrete repository : `SqlAlchemyRepository`" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "from typing import Optional, Callable\n", "from sqlalchemy.orm import Session\n", "\n", "class SqlAlchemyRepository(AbstractRepository):\n", " \n", " def __init__(self, session: Session):\n", " self.session = session\n", " \n", " def close(self):\n", " self.session.close()\n", " \n", " def add(self, batch: Batch):\n", " self.session.add(batch)\n", " self.session.commit()\n", "\n", " def get(self, reference) -> Optional[Batch]:\n", " return self.session.query(Batch).filter_by(reference=reference).first()\n", " \n", " def list(self):\n", " return self.session.query(model.Batch).all() \n", " \n", " def clear(self):\n", " self.session.execute('DELETE FROM allocation')\n", " self.session.execute('DELETE FROM batch')\n", " self.session.execute('DELETE FROM order_line')\n", " self.session.commit()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Repository test for saving an object" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ \u001b[95mtest_repository_can_save_a_batch\u001b[0m\n" ] } ], "source": [ "@mytest.unit\n", "def test_repository_can_save_a_batch(session):\n", " with session as db:\n", " repo = SqlAlchemyRepository(db)\n", " try:\n", " batch = Batch(\"batch1\", \"RUSTY-SOAPDISH\", 100, eta=None)\n", " repo.add(batch)\n", " rows = list(db.execute(\n", " 'SELECT reference, sku, _purchased_quantity, eta FROM \"batch\"'\n", " ))\n", " assert rows == [(\"batch1\", \"RUSTY-SOAPDISH\", 100, None)]\n", " finally:\n", " repo.clear()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Repository test for retrieving a complex object " ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ \u001b[95mtest_repository_can_retrieve_a_batch_with_allocations\u001b[0m\n" ] } ], "source": [ "def insert_order_line(db, orderid=\"order1\", sku=\"GENERIC-SOFA\"):\n", " db.execute(\n", " 'INSERT INTO order_line (orderid, sku, qty) '\n", " 'VALUES (:orderid, :sku, 12)',\n", " dict(orderid=\"order1\", sku=\"GENERIC-SOFA\")\n", " )\n", " [orderline_id], *_ = db.execute(\n", " 'SELECT id FROM order_line WHERE orderid=:orderid AND sku=:sku',\n", " dict(orderid=\"order1\", sku=\"GENERIC-SOFA\")\n", " )\n", " return orderline_id\n", "\n", "\n", "def insert_batch(db, reference, sku=\"GENERIC-SOFA\", qty=100):\n", " db.execute(\n", " 'INSERT INTO batch (reference, sku, _purchased_quantity) '\n", " 'VALUES (:reference, :sku, :qty)',\n", " dict(reference=reference, sku=sku, qty=qty)\n", " )\n", " [batch_id], *_ = db.execute(\n", " 'SELECT id FROM batch WHERE reference=:reference',\n", " dict(reference=reference)\n", " )\n", " return batch_id\n", "\n", "\n", "def insert_allocation(db, orderline_id, batch_id):\n", " db.execute(\n", " 'INSERT INTO allocation (orderline_id, batch_id) '\n", " 'VALUES (:orderline_id, :batch_id)',\n", " dict(orderline_id=orderline_id, batch_id=batch_id)\n", " )\n", "\n", "@mytest.unit\n", "def test_repository_can_retrieve_a_batch_with_allocations(session):\n", " with session as db:\n", " repo = SqlAlchemyRepository(db)\n", " try:\n", " orderline_id = insert_order_line(db)\n", " batch1_id = insert_batch(db, \"batch1\", qty=100)\n", " batch2_id = insert_batch(db, \"batch2\")\n", " insert_allocation(db, orderline_id, batch1_id)\n", "\n", " retrieved = repo.get(\"batch1\")\n", " expected = Batch(\"batch1\", sku=\"GENERIC-SOFA\", qty=100, eta=None)\n", "\n", " assert retrieved == expected # Batch.__eq__ only compares reference\n", " assert retrieved.sku == expected.sku\n", " assert retrieved._purchased_quantity == expected._purchased_quantity\n", " assert retrieved._allocations == {\n", " OrderLine(\"order1\", \"GENERIC-SOFA\", 12),\n", " }\n", " finally:\n", " repo.clear()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Building a Fake Repository for Tests is Now Trivial!" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "class FakeRepository(AbstractRepository):\n", "\n", " def __init__(self, batches):\n", " self._batches = set(batches)\n", "\n", " def close(self):\n", " pass\n", " \n", " def add(self, batch):\n", " self._batches.add(batch)\n", " \n", " def get(self, reference):\n", " return next(b for b in self._batches if b.reference == reference)\n", "\n", " def list(self):\n", " return list(self._batches)\n", " \n", " def clear(self):\n", " self._batches = set()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Example usage of fake repository " ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "def fake_repo():\n", " batch1 = Batch(\"batch1\", sku=\"TEST-TABLE\", qty=100)\n", " batch2 = Batch(\"batch2\", sku=\"TEST-CHAIR\", qty=50)\n", " batch3 = Batch(\"batch3\", sku=\"TEST-CABLE\", qty=10)\n", " return FakeRepository([batch1, batch2, batch3])\n", "\n", "def fake_request_params():\n", " return [\n", " dict(orderid=\"order1\", sku=\"TEST-TABLE\", qty=10),\n", " dict(orderid=\"order2\", sku=\"TEST-TABLE\", qty=20),\n", " dict(orderid=\"order3\", sku=\"TEST-CHAIR\", qty=30),\n", " dict(orderid=\"order4\", sku=\"TEST-CABLE\", qty=10),\n", " ]" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "✅ \u001b[95mtest_fake_repo\u001b[0m\n" ] } ], "source": [ "from app.services import allocate\n", "\n", "@mytest.unit\n", "def test_fake_repo(fake_repo, session, fake_request_params):\n", " with session as db:\n", " batches = fake_repo.list()\n", " lines = [\n", " OrderLine(l['orderid'], l['sku'], l['qty'])\n", " for l in fake_request_params\n", " ]\n", " \n", " for line in lines:\n", " allocate(line, batches)\n", " db.commit() " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## What is a Port and What is an Adapter, in Python?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ports and adapters came out of the OO world\n", "- **port** : an interface between our application and whatever we wish to abstract away\n", "- **adapter** : an implementation behind that interface or abstraction.\n", "\n", "Python doesn't have interfaces, so defining a port can be simulated with `abc.ABC` or `Protocol`.\n", "\n", "In this chapter,\n", "- port : `AbstractRepository`.\n", "- adapters : `SqlAlchemyRepository` and `FakeRepository`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Wrap-Up" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Summerize the costs and benefits of each architectural pattern we introduce. \n", "- Every single application *DO NOT* need to be built this way\n", " - Make sure the overall complexity of the repository pattern makes it worth in adding extra layers of indirection.\n", " - If your app is just a simple CRUD wrapper around a database, then YOU DON'T NEED a domain model or a repository." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Repository pattern and persistence ignorance: the trade-offs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
ProsCons
\n", "
    \n", "
  • We have a simple interface between persistent storage and our domain model.
    • \n", "
  • It’s easy to make a fake version of the repository for unit testing, or to swap out different storage solutions, because we’ve fully decoupled the model from infrastructure concerns.
    • \n", "
  • Writing the domain model before thinking about persistence helps us focus on the business problem at hand. If we ever want to radically change our approach, we can do that in our model, without needing to worry about foreign keys or migrations until later.
    • \n", "
  • Our database schema is really simple because we have complete control over how we map our objects to tables.
    • \n", "
\n", "		\n", "
    \n", "
  • An ORM already buys you some decoupling. Changing foreign keys might be hard, but it should be pretty easy to swap between MySQL and Postgres if you ever need to.
    • \n", "
  • Maintaining ORM mappings by hand requires extra work and extra code.
    • \n", "
  • Any extra layer of indirection always increases maintenance costs and adds a \"WTF factor\" for Python programmers who’ve never seen the Repository pattern before.
    • \n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![](images/apwp_0206.png)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Repository Pattern Recap" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### Apply dependency inversion to your ORM\n", "\n", "Our domain model should be free of infrastructure concerns, so your ORM should import your model, and not the other way around.\n", "\n", "##### The Repository pattern is a simple abstraction around permanent storage\n", "\n", "The repository gives you the illusion of a collection of in-memory objects. It makes it easy to create a FakeRepository for testing and to swap fundamental details of your infrastructure without disrupting your core application. See [appendix_csvs] for an example." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.1" } }, "nbformat": 4, "nbformat_minor": 4 }