=========
Odoo svil
=========

This section cover all concepts regarding the odoo svil workflow.
Please read before the introduction in :doc:`index`.

Useful sections (for those unable to read documentation):

* **Use docker-compose with odoo**: `Docker-compose`_
* **Dump and restore databases**: `Dump/restore the production or demo database`_

Architecture
============

To reproduce a minimal architecture to test and develop odoo modules on local machine we use ``docker-compose`` utility.
A generic ``docker-compose`` stack for odoo contains 2 services (every service generate 1 container): 

* **odoo** 
* **postgres**.

This is an axample of ``docker-compose.yml`` file used for odoo:

.. code-block:: yaml

        version: '3'
        services:
          postgres:
            image: postgres:9.6
            environment:
              PGDATA: /var/lib/postgresql/data
              POSTGRES_USER: odoo
              POSTGRES_PASSWORD: odoo
              POSTGRES_DB: postgres
            ports:
              - 5000:5432
            volumes:
              - db-data:/var/lib/postgresql/data
            restart: 'no'
          odoo:
            build: .
            image: odoo:base-13
            volumes:
              - odoo-data:/var/lib/odoo
              - ./odoo.conf:/etc/odoo/odoo.conf
              - ./addons:/mnt/extra-addons:ro
            ports:
              - 8069:8069
              - 8072:8072
            depends_on:
              - postgres
            restart: 'no'
            stdin_open: true
            tty: true
        volumes:
          db-data:
            driver: local
          odoo-data:
            driver: local


Common operations
=================

Docker-compose
--------------

To run docker-compose stack using ``docker-compose.yml`` file simply issue:

.. code-block:: sh

   ~$ docker-compose up

if you want to specify other files (for ex. development.yml) to docker-compose issue:

.. code-block:: sh

   ~$ docker-compose -f docker-compose.yml -f development.yml up

You can also use ``run`` argument instead ``up`` to pass arguments to a specific service entrypoint and to open an interactive tty. For ex.:

.. code-block:: sh

   ~$ docker-compose run --rm --service-ports odoo -- <arguments>

For example when developing a module and making changes, especially to the database, it will be necessary to restart odoo.
If the system was launched in development mode, simply refresh the page with the option `--dev=all`

* for only xml modifications it is possible to do:

   .. code-block:: sh
   
      $ docker-compose run --rm \
        --service-ports odoo -- -d 'DATABASE_NAME' \
        --dev=xml -u MODULE_NAME


* to update a module from the shell it is possible to do:

   .. code-block:: sh
   
      $ docker-compose run --rm \
        --service-ports odoo -- -d 'DATABASE_NAME' \
        -u MODULE_NAME


* if you want to restore a dump file see the section below



Dump/restore the production or demo database
--------------------------------------------
It may often be useful to develop an updated copy of the production database
locally so as to have real and updated data.
To do this you can use the appropriate postgres clients
(``pg_dump``, ``pg_restore``), even without needing to install them on the
machine, it can be used directly from the postgres container.

.. important::
        
   These steps works for svil (local) docker environment

To dump the database:

.. code-block:: sh

   $ docker exec -i <nome-container-postgres> pg_dump \
     --format=custom -U odoo -d <database-name> > <output-file>

To restore it:

.. code-block:: sh

   $ docker exec -i <nome-container-postgres> pg_restore \
     -U odoo -O -d <database-name> < <database-dump>

.. important::

   These steps works for demo environment

* Login into ``docker-be-svil-manager01``
* First identify demo database user and pw.
* Login into ``dbcl-preprod.localmonks.net``
* Create ``.pgpass`` file in your home
* and place inside ``<hostname>:<port>:<database>:<username>:<password>`` that you find on ``docker-be-svil-manager01``
* set ``chmod 600 .pgpass``
* run the command ``pg_dump --format=custom -h hostname -U user -d database > output-file``
* scp the dump from server to your local machine

.. caution::

   Remember to reset some "dangerous" production data just after database recovery.:

   * Outgoing/incoming mail servers (to not send/receive real mails) on all

   .. code-block:: sql

      DELETE FROM fetchmail_server;
      DELETE FROM ir_mail_server;

   * The web.base.url parameter (odoo URL, used for example in links sent by email)

.. note::

   Unless you also import a filestore backup, it is recommended 
   that you also delete all the `` ir_attachment`` records to avoid references to non-existent files.

   .. code-block:: sql

      DELETE FROM ir_attachment;

Debugging
---------

During the development of a backend model, it is possible to debug with pdb:

.. code-block:: python

   import pdb; pdb.set_trace()


The `set_trace()` method can stop odoo at the desired point.

Then launch the command:

.. code-block:: sh

   $ docker-compose run --rm --service-ports odoo


at this point odoo will stop and the shell debugging phase will begin.

.. _on the manifest file : https://www.odoo.com/documentation/13.0/reference/module.html

Odoo developer mode
-------------------

To activate it, go to the 'Settings' application and press 'Activate the developer mode' on the right, under the Odoo version.

otherwise...

add this line to the local host: `http://localhost:8069/web?debug#`


**In odoo 13, there is a little change**: `http://localhost:8069/web?debug=1#`


We can go back to the App list. At this point you can see the 'Update App list' button.
Pressing the button will perform the search of the module that would be now able to install it.


Scaffolding Odoo Module
=======================

Module skeleton 
---------------

Odoo has its own scaffolding mechanism to create new modules
it is possible to see this operation by launching this command 
from the folder containing the odoo environment:

.. code-block:: sh

   $ docker-compose run --rm odoo -- scaffold --help


It can be useful when you have confidence with the creation of a module
but in this documentation we are going to create the structure manually,
in order to understand the various steps.


Structure
---------

An odoo module is a directory that has the following files:
__manifest__.py and __init__.py.

The directory name module it is a technical valid name in python,
so it is mandatory to insert letters, numbers and underscore.
In our case we use this example name **monk_app**.

Inside the main directory of the project, you can find the **addons** directory. 
Within this latter you find the **sources** directory and you create the monk_app with the following structure:

.. code-block::

   monk_app/
      __init__.py
      __manifest__.py

.. important::

   Remember to add a symlink into the enabled directory like this: ``ln -s ../addons/sources/monk_app``

Inside the __manifest__.py you have to add:

.. code-block:: python

   {
      'name': 'Monk Application',
      'version': '1.0',
      'description': '...',
      'author': 'Monk Software',
      'depends': ['base'],
      'data': [

      ],
      'application': True,
      'installable': True,
   }

The **depends** field indicates the modules on which our application depends.

The field must be add especially when referring to third-party features.

For a more detailed explanation of __manifest__.py go to the page of the Odoo documentation `on the manifest file`_


Addon's path
------------

In our case Odoo will be able to find our application because we have created it in the environment addons folder
To set the different paths where odoo looks for available addons, configuration and storage data,
you need to configure the file docker-compose.yml:

.. code-block:: yaml

   volumes:
      - odoo-data:/var/lib/odoo
      - ./odoo.conf:/etc/odoo/odoo.conf
      - ./addons:/mnt/extra-addons:ro


Module installation
-------------------

At this point it is possible to install the module.
We can go to the App list, write **monk_app** in the search bar.

But... the module doesn't appear, even though it seems that everything was done right.

To make our module appear we need to reload the application list.
To do that we have to activate the `Odoo developer mode`_

Advice and utilities
====================

Pull the latest version of odoo image
-------------------------------------

To pull the latest version of the desire odoo image, run this command:

   :: 
      
      docker pull odoo:12.0

obviously you can choose the version of odoo that you want.

Unit tests
----------

The unit tests are performed from odoo only on databases and modules with the demo data installed.
The best way to perform the tests is therefore to create a
dedicated database with demo data and run tests on that.
You can do everything in one command by launching, for example ::

   docker-compose run --rm odoo -- --workers=0 --stop-after-init --test-enable
   -d <nome_database_per_i_test> --without-demo="" -i <nome_modulo_da_testare>

This command will create a new database with the specified name, 
installing the chosen module and all its dependencies with the demo data (therefore testable).
It then performs the tests of the installed modules and exits after having finished.

Once the demo database for the tests has been created, 
from the second time onwards it is enough to upgrade the module you want to test ::

   docker-compose run --rm odoo -- --workers=0 --stop-after-init --test-enable
   -d <nome_database_per_i_test> -u <nome_modulo_da_testare>


Useful modules
==============

* `auth_brute_force`_: Set a limit of incorrect login attempts
  to block passwords bruteforce attempts (by default odoo does not set any limits).
  It works both on the standard login form and via API methods.
  It would be installed in every instance in production to increase security.

* `auth_admin_passkey`_: It allows you to log in to odoo like any user using the admin password. 
  It is useful for testing as various users without having to go and ask or change users' real passwords,
  or having to create special users for testing.

* `database_cleanup`_: Odoo by default does not delete anything from the db,
  not even existing columns or other data removed or useless, 
  to avoid losing data that maybe you want to recover in the future. 
  There is no problem to leave "the remains" in the db, but in particular cases or if it really bothers you,
  you can use this form to facilitate the identification and removal of the remains.

.. _auth_brute_force: https://github.com/OCA/server-tools/tree/10.0/auth_brute_force
.. _auth_admin_passkey: https://github.com/OCA/server-tools/tree/10.0/auth_admin_passkey
.. _database_cleanup: https://github.com/OCA/server-tools/tree/10.0/database_cleanup


Useful solutions
================

Sometimes it happened that the image of odoo 12 gave some errors, giving problems on the initialization of the database.

you should see something like this in your shell:

::

   odoo_1      | return self.models[model_name]
   odoo_1      | KeyError: 'ir.http' - -

To fix temporarily this problem, you can inizialize the database in another shell!

First, run the service in your shell:

::

   docker-compose -f docker-compose.development.yml -f docker-compose.yml up

Then, open another shell and run this command

::

   docker-compose exec odoo bash

And then, inizialize the database:

::

   odoo -i base -d odoo --stop-after-init --db_host=db -r odoo -w odoo

This command will automatically create a database called 'odoo'. Now, you can quit the second shell and work without problem.


Do you prefer to avoid this situation every time? Here there is another solution to fix this problem:


1) create a file called entrypoint.sh in the same parent directory

.. code:: sh

         #!/bin/bash

         set -e

         # set the postgres database host, port, user and password according to the environment
         # and pass them as arguments to the odoo process if not present in the config file
         : ${HOST:=${DB_PORT_5432_TCP_ADDR:='db'}}
         : ${PORT:=${DB_PORT_5432_TCP_PORT:=5432}}
         : ${USER:=${DB_ENV_POSTGRES_USER:=${POSTGRES_USER:='odoo'}}}
         : ${PASSWORD:=${DB_ENV_POSTGRES_PASSWORD:=${POSTGRES_PASSWORD:='odoo'}}}

         DB_ARGS=()
         function check_config() {
            param="$1"
            value="$2"
            if grep -q -E "^\s*\b${param}\b\s*=" "$ODOO_RC" ; then       
               value=$(grep -E "^\s*\b${param}\b\s*=" "$ODOO_RC" |cut -d " " -f3|sed 's/["\n\r]//g')
            fi;
            DB_ARGS+=("--${param}")
            DB_ARGS+=("${value}")
         }
         check_config "db_host" "$HOST"
         check_config "db_port" "$PORT"
         check_config "db_user" "$USER"
         check_config "db_password" "$PASSWORD"

         case "$1" in
            -- | odoo)
               shift
               if [[ "$1" == "scaffold" ]] ; then
                     echo "case 1"
                     wait-for-psql.py ${DB_ARGS[@]} --timeout=30
                     ./odoo-postgres-init.sh
                     exec odoo "$@"

               else
                     echo "case 2"
                     wait-for-psql.py ${DB_ARGS[@]} --timeout=30
                     #init database work in progress
                     ./odoo-postgres-init.sh
                     exec odoo "$@" "${DB_ARGS[@]}"
               fi
               ;;
            -*)
               echo "case 3"
               wait-for-psql.py ${DB_ARGS[@]} --timeout=30
               #init database work in progress
               ./odoo-postgres-init.sh
               exec odoo "$@" "${DB_ARGS[@]}"
               ;;
            *)
               exec "$@"
         esac

         exit 1


2) create an odoo-postgres-init.sh file in the same directory

::

   odoo -i base -d odoo --stop-after-init --db_host=db -r odoo -w odoo


3) edit the Dockerfile

::

   FROM odoo:12.0
   MAINTAINER MONK SOFTWARE <http://monksoftware.it>
   USER root
   COPY requirements.txt /tmp/
   RUN pip3 install -r /tmp/requirements.txt
   COPY ./odoo-postgres-init.sh ./odoo-postgres-init.sh
   RUN chmod 644 ./odoo-postgres-init.sh
   COPY ./entrypoint.sh ./entrypoint.sh
   RUN chmod 644 ./entrypoint.sh
   USER odoo
   COPY ./addons /mnt/extra-addons


4) build the image

::
   
   docker-compose -f docker-compose.yml -f docker-compose.development.yml build --no-cache # build image

5) Then, run docker

::

   docker-compose -f docker-compose.yml -f docker-compose-development.yml up