Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
244 changes: 8 additions & 236 deletions src/7-to-8/major-changes/compatibility-mode.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,241 +3,13 @@
Cylc 7 Compatibility Mode
=========================

.. warning::
Cylc 8 initially provided a Cylc 7 compatibility mode which allowed workflows
to be run under either Cylc 7 or 8, to help facilitate migration.

Cylc 7 compatibility mode will be removed in Cylc 8.7.0, please upgrade
any workflows with "suite.rc" files.
This compatibility mode was removed in Cylc 8.7.0. If you have any workflows
that still use a ``suite.rc`` file, please skim through
:ref:`configuration-changes` then rename this file to ``flow.cylc`` and address
any errors/warnings.

.. admonition:: Does This Change Affect Me?
:class: tip

This will affect you if you want to run Cylc 7 (``suite.rc``) workflows
using Cylc 8.

Overview
--------

Cylc 8 can run most Cylc 7 workflows "as is".
The ``suite.rc`` filename triggers a backward compatibility mode in which:

- :term:`implicit tasks <implicit task>` are allowed by default

- (unless a ``rose-suite.conf`` file is found in the :term:`run directory`
for consistency with ``rose suite-run`` behaviour)
- (Cylc 8 does not allow implicit tasks by default)

- :term:`cycle point time zone` defaults to the local time zone

- (Cylc 8 defaults to UTC)

- waiting tasks are pre-spawned to mimic the Cylc 7 scheduling algorithm and
stall behaviour, and these require
:term:`suicide triggers <suicide trigger>`
for alternate :term:`graph branching`

- (Cylc 8 spawns tasks on demand, and suicide triggers are not needed for
branching)

- ``succeeded`` task outputs are :ref:`required <User Guide Required Outputs>`,
so in the absence of suicide triggers the scheduler will retain other
:term:`final status` tasks in the :term:`n=0 window <n-window>` to stall the
workflow.

- (in Cylc 8, **all** outputs are *required* unless marked as
:ref:`*optional* <User Guide Optional Outputs>` by the new ``?`` syntax)


.. _compat_required_changes:

Required Changes
----------------

Providing your Cylc 7 workflow does not use syntax that was deprecated at Cylc 7,
you may be able to run it using Cylc 8 without any modifications while in
compatibility mode.

First, run ``cylc validate`` **with Cylc 7** on your ``suite.rc`` workflow
to check for deprecation warnings and fix those before validating with Cylc 8.
See :ref:`below <compat.eg.c7val>` for an example.

.. warning::

``cylc validate`` operates on the processed ``suite.rc``, which
means it will not detect any deprecated syntax that is inside a
currently-unused Jinja2 ``if...else`` branch.

Some workflows may require modifications to either upgrade to Cylc 8 or make
interoperable with Cylc 8 backward compatibility mode. Read on for more details.


Cylc commands in task scripts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Check for any use of Cylc commands in task scripting. Some Cylc 7 commands
have been removed and some others now behave differently.
However, ``cylc message`` and ``cylc broadcast`` have *not* changed.
See the :ref:`full list of command line interface changes<MajorChangesCLI>`
and see :ref:`below <compat.eg.cylc-commands>` for an example.


Python 2 to 3
^^^^^^^^^^^^^

Whereas Cylc 7 runs using Python 2, Cylc 8 runs using Python 3. This affects:
- modules imported in Jinja2
- Jinja2 filters, tests and globals
- custom xtrigger functions

Note that task scripts are not affected - they run in an independent
environment.

See :ref:`py23` for more information and examples of how to implement
interoperability if your workflows extend Cylc or Jinja2 with custom Python scripts.


Other caveats
^^^^^^^^^^^^^

- Cylc 8 cannot *restart* a partially completed Cylc 7 workflow in-place. If
possible, complete the run with Cylc 7. Otherwise, see
:ref:`compat_continuing_c7_with_c8`.

- Cylc 8 only transfers certain files and directories by default during
remote installation. See :ref:`728.remote-install` for more information.

- Cylc 8 does not support
:ref:`excluding/including tasks at start-up<MajorChangesExcludingTasksAtStartup>`.
If your workflow used this old functionality, it may have been used in
combination with the ``cylc insert`` command (which has been removed from
Cylc 8) and ``cylc remove`` (which still exists but is much less needed).

- Cylc 8 does not support :ref:`specifying remote usernames <728.remote_owner>`
using :cylc:conf:`flow.cylc[runtime][<namespace>][remote]owner`.


Examples
--------

.. _compat.eg.c7val:

Validating with Cylc 7
^^^^^^^^^^^^^^^^^^^^^^

Consider this configuration:

.. code-block:: cylc
:caption: ``suite.rc``

[scheduling]
initial cycle point = 11000101T00
[[dependencies]]
[[[R1]]]
graph = task

[runtime]
[[task]]
pre-command scripting = echo "Hello World"

Running ``cylc validate`` at **Cylc 7** we see that the
workflow is valid, but we are warned that ``pre-command scripting``
was replaced by ``pre-script`` at 6.4.0:

.. code-block:: console
:caption: Cylc 7 validation

$ cylc validate .
WARNING - deprecated items were automatically upgraded in 'suite definition':
WARNING - * (6.4.0) [runtime][task][pre-command scripting] -> [runtime][task][pre-script] - value unchanged
Valid for cylc-7.8.7

.. note::

**Cylc 7** has handled this deprecation for us, but at **Cylc 8** this
workflow will fail validation.

.. code-block:: console
:caption: Cylc 8 validation

$ cylc validate .
IllegalItemError: [runtime][task]pre-command scripting

You must change the configuration yourself. In this case:

.. code-block:: diff

- pre-command scripting = echo "Hello World"
+ pre-script = echo "Hello World"

Validation will now succeed.


.. _compat.eg.cylc-commands:

Cylc commands in task scripts
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You might have a task script that calls a Cylc command like so:

.. code-block:: cylc

[runtime]
[[foo]]
script = cylc hold "$CYLC_SUITE_NAME"

The ``cylc hold`` command has changed in Cylc 8. It is now used for holding
tasks only; use ``cylc pause`` for entire workflows.
(Additionally, ``$CYLC_SUITE_NAME`` is deprecated in favour of
:envvar:`CYLC_WORKFLOW_ID`, though still supported.)

In order to make this interoperable, so that you can run it with both Cylc 7
and Cylc 8 backward compatibility mode, you could do something like this
in the bash script:

.. code-block:: cylc

[runtime]
[[foo]]
script = """
if [[ "${CYLC_VERSION:0:1}" == 7 ]]; then
cylc hold "$CYLC_SUITE_NAME"
else
cylc pause "$CYLC_WORKFLOW_ID"
fi
"""

Note this logic (and the :envvar:`CYLC_VERSION` environment variable) is
executed at runtime on the :term:`job host`.

Alternatively, you could use :ref:`Jinja` like so:

.. code-block:: cylc

[runtime]
[[foo]]
{% if CYLC_VERSION is defined and CYLC_VERSION[0] == '8' %}
script = cylc pause "$CYLC_WORKFLOW_ID"
{% else %}
script = cylc hold "$CYLC_SUITE_NAME"
{% endif %}

Note this logic (and the :envvar:`CYLC_VERSION` Jinja2 variable) is executed
locally prior to Cylc parsing the workflow configuration.


Renaming to ``flow.cylc``
-------------------------

When your workflow runs successfully in backward compatibility mode, it is
ready for renaming ``suite.rc`` to ``flow.cylc``. Doing this will turn off
backward compatibility mode, and validation in Cylc 8 will show
deprecation warnings.

.. seealso::

:ref:`configuration-changes`

.. important::

More complex workflows (e.g. those with suicide triggers) may
fail validation once backward compatibility is off - see
:ref:`728.optional_outputs`
For more information on what compatibility mode did, please see the
`Cylc 8.6 documentation <https://cylc.github.io/cylc-doc/8.6.3/html/7-to-8/major-changes/compatibility-mode.html>`_.
13 changes: 0 additions & 13 deletions src/7-to-8/major-changes/cylc-install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -99,19 +99,6 @@ However, Cylc 8 will only copy the ``ana``, ``app``, ``bin``, ``etc`` and
If you want to include custom files and directories in remote installation,
use :cylc:conf:`flow.cylc[scheduler]install`.

.. tip::

If you need to ensure your workflow is still
:ref:`interoperable <cylc_7_compat_mode>` with Cylc 7, wrap it in a
Jinja2 check like so:

.. code-block:: cylc

{% if CYLC_VERSION is defined and CYLC_VERSION[0] == '8' %}
[scheduler]
install = my-dir/, my-file
{% endif %}

See :ref:`the user guide <RemoteInit>` for more details.

.. warning::
Expand Down
8 changes: 0 additions & 8 deletions src/7-to-8/major-changes/platforms.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,6 @@ suitable for a task. Such platforms can be set up to be ``platform groups``
:ref:`AdminGuide.PlatformConfigs` for detailed examples of platform
configurations.

.. tip::

Cylc 8 contains upgrade logic (:ref:`see below <host-to-platform-logic>`)
which handles the deprecated Cylc 7 settings in most cases.
Unless you are in :ref:`backward compatibility mode <cylc_7_compat_mode>`,
you should upgrade to using platforms instead.
Deprecated settings will be removed in a later release of Cylc.


What is a Platform?
-------------------
Expand Down
3 changes: 0 additions & 3 deletions src/7-to-8/major-changes/suicide-triggers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ the need for suicide triggers). This is called :term:`graph branching`.
:term:`suicide triggers <suicide trigger>` (marked by ``!`` before the
task name in the graph, e.g. ``foo:fail => !foo``).

You should *not* perform this upgrade if still in :ref:`cylc_7_compat_mode`
(``suite.rc`` filename).


Required Changes
^^^^^^^^^^^^^^^^
Expand Down
20 changes: 1 addition & 19 deletions src/7-to-8/summary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,6 @@ Cylc now uses more widely understood terms for several core concepts.

Note the configuration filename is now ``flow.cylc``, not ``suite.rc``.

Cylc 7 Compatibility Mode
-------------------------

Continuing to use the old ``suite.rc`` filename triggers a :ref:`backward
compatibility mode<cylc_7_compat_mode>` in Cylc 8 which supports Cylc 7
workflow configurations out of the box, with
:ref:`some caveats <compat_required_changes>`. However, to future-proof
your workflow and take full advantage of Cylc 8 you should upgrade to Cylc 8 syntax.

.. warning::

Cylc 7 compatibility mode will be removed in Cylc 8.7.0.


Upgrading To Cylc 8
-------------------
.. seealso::
Expand All @@ -47,9 +33,6 @@ To upgrade your Cylc 7 suite to a Cylc 8 workflow:

#. Using Cylc 7, make sure the configuration validates (``cylc validate``)
without any warnings.
#. Using Cylc 8 check that you can run the workflow. Running Cylc 8 with a
workflow configured with a ``suite.rc`` turns on
:ref:`compatibility mode <cylc_7_compat_mode>`.
#. Rename the workflow configuration file from ``suite.rc`` to ``flow.cylc``.
#. Using Cylc 8 run ``cylc lint --ruleset 728`` and ``cylc validate``. Make
sure that you deal with any warnings produced by these scripts.
Expand Down Expand Up @@ -290,8 +273,7 @@ Queues
we now recommend using queues to restrict the number of running tasks in
situations where graphing may have been used previously.
Time Zones
:cylc:conf:`[scheduler]cycle point time zone` now defaults to UTC, unless you
are working in :ref:`cylc_7_compat_mode`.
:cylc:conf:`[scheduler]cycle point time zone` now defaults to UTC.
Job Scripts
All user-defined task scripting now runs in a subshell, so you can safely
switch Python environments inside tasks without affecting Cylc.
Expand Down
1 change: 1 addition & 0 deletions src/dictionaries/words
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ baz
boolean
booleans
broadcasted
bugfixes
cfg
changelog
changelogs
Expand Down
25 changes: 25 additions & 0 deletions src/reference/changes.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,35 @@
.. _reference.changes:

Changes
=======

This page contains a summary of significant changes across all Cylc components for each
release.


Cylc 8.7
--------


.. admonition:: Cylc Components
:class: hint

TODO!


Cylc 7 Compatibility Mode
^^^^^^^^^^^^^^^^^^^^^^^^^

The Cylc 7 compatibility mode (which allowed workflows to run under both Cylc 7
and 8) has now been removed.

See :ref:`cylc_7_compat_mode` for more details.

As part of this change, the version of Jinja2 that Cylc uses has increased From
3.0 to 3.1+. As a result, any use of deprecated Jinja2 interfaces will likely
break with this Cylc release.


Cylc 8.6
--------

Expand Down
3 changes: 0 additions & 3 deletions src/tutorial/scheduling/graphing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,6 @@ The :cylc:conf:`flow.cylc` File Format
Prior to Cylc 8, :cylc:conf:`flow.cylc` was named ``suite.rc``,
but that name is now deprecated.

See :ref:`cylc_7_compat_mode` for information on compatibility with
existing Cylc 7 ``suite.rc`` files.

Example
^^^^^^^

Expand Down
1 change: 1 addition & 0 deletions src/user-guide/running-workflows/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ Running Workflows
authentication-files
workflow-databases
advanced
inter-version-compatibility
Loading
Loading