Sanity¶

The Sanity task is a software watchdog task, which runs periodically to check system state, and ensure that everything is still operating properly.

In a typical system design, there are multiple stages of watchdog:

Internal Watchdog
External Watchdog
Sanity Watchdog

The Internal Watchdog is typically an MCU watchdog, which is tickled in the core of the OS. The internal watchdog is tickled frequently, and is meant to be an indicator the OS is running.

The External Watchdog is a watchdog that’s typically run slower. The purpose of an external watchdog is to provide the system with a hard reset when it has lost its mind.

The Sanity Watchdog is the least frequently run watchdog, and is meant as an application watchdog.

This document is about the operation of the Mynewt Sanity Watchdog.

Description¶

Sanity Task¶

Mynewt OS uses the OS Idle task to check sanity. The SANITY_INTERVAL syscfg setting specifies the interval in seconds to perform the sanity checks.

By default, every operating system task provides the frequency it will check in with the sanity task, with the sanity_itvl parameter in the c:func:os_task_init() function:

int os_task_init(struct os_task *t, char *name, os_task_func_t func,
    void *arg, uint8_t prio, os_time_t sanity_itvl, os_stack_t *bottom,
    uint16_t stack_size);

c:var:sanity_itvl is the time in OS time ticks that the task being created must register in with the sanity task.

Checking in with Sanity Task¶

The task must then register in with the sanity task every sanity_itvl seconds. In order to do that, the task should call the os_sanity_task_checkin() function, which will reset the sanity check associated with this task. Here is an example of a task that uses a callout to checkin with the sanity task every 50 seconds:

#define TASK1_SANITY_CHECKIN_ITVL (50 * OS_TICKS_PER_SEC)
struct os_eventq task1_evq;

static void
task1(void *arg)
{
    struct os_task *t;
    struct os_event *ev;
    struct os_callout c;

    /* Get current OS task */
    t = os_sched_get_current_task();

    /* Initialize the event queue. */
    os_eventq_init(&task1_evq);

    /* Initialize the callout */
    os_callout_init(&c, &task1_evq, NULL);

    /* reset the callout to checkin with the sanity task
     * in 50 seconds to kick off timing.
     */
    os_callout_reset(&c, TASK1_SANITY_CHECKIN_ITVL);

    while (1) {
        ev = os_eventq_get(&task1_evq);

        /* The sanity timer has reset */
        if (ev->ev_arg == &c) {
            os_sanity_task_checkin(t);
        } else {
            /* not expecting any other events */
            assert(0);
        }
    }

    /* Should never reach */
    assert(0);
}

Registering a Custom Sanity Check¶

If a particular task wants to further hook into the sanity framework to perform other checks during the sanity task’s operation, it can do so by registering a struct os_sanity_check using the os_sanity_check_register() function.

static int
mymodule_perform_sanity_check(struct os_sanity_check *sc, void *arg)
{
    /* Perform your checking here.  In this case, we check if there
     * are available buffers in mymodule, and return 0 (all good)
     * if true, and -1 (error) if not.
     */
    if (mymodule_has_buffers()) {
        return (0);
    } else {
        return (-1);
    }
}

static int
mymodule_register_sanity_check(void)
{
    struct os_sanity_check sc;

    os_sanity_check_init(&sc);
    /* Only assert() if mymodule_perform_sanity_check() fails 50
     * times.  SANITY_TASK_INTERVAL is defined by the user, and
     * is the frequency at which the sanity_task runs in seconds.
     */
    OS_SANITY_CHECK_SETFUNC(&sc, mymodule_perform_sanity_check, NULL,
        50 * SANITY_TASK_INTERVAL);

    rc = os_sanity_check_register(&sc);
    if (rc != 0) {
        goto err;
    }

    return (0);
err:
    return (rc);
}

In the above example, every time the custom sanity check mymodule_perform_sanity_check returns successfully (0), the sanity check is reset. In the c:macro:OS_SANITY_CHECK_SETFUNC macro, the sanity checkin interval is specified as 50 * SANITY_TASK_INTERVAL (which is the interval at which the sanity task runs.) This means that the mymodule_perform_sanity_check() function needs to fail 50 times consecutively before the sanity task will crash the system.

TIP: When checking things like memory buffers, which can be temporarily be exhausted, it’s a good idea to have the sanity check fail multiple consecutive times before crashing the system. This will avoid crashing for temporary failures.