vim-jp / vimdoc-en / testing

testing - Vim Documentation

Return to main English | 日本語
testing.txt   For Vim version 9.1.  Last change: 2024 Jul 18


                  VIM REFERENCE MANUAL    by Bram Moolenaar


Testing Vim and Vim script                      testing-support

Expression evaluation is explained in eval.txt.  This file goes into details
about writing tests in Vim script.  This can be used for testing Vim itself
and for testing plugins.

1. Testing Vim                          testing
2. Test functions                       test-functions-details
3. Assert functions                     assert-functions-details

==============================================================================
1. Testing Vim                                          testing

Vim can be tested after building it, usually with "make test".
The tests are located in the directory "src/testdir".

There are two types of tests added over time:
        test20.in               oldest, only for tiny builds
        test_something.vim      new style tests

                                                new-style-testing
New tests should be added as new style tests.  The test scripts are named
test_<feature>.vim (replace <feature> with the feature under test). These use
functions such as assert_equal() to keep the test commands and the expected
result in one place.
                                                old-style-testing
These tests are used only for testing Vim without the +eval feature.

Find more information in the file src/testdir/README.txt.

==============================================================================
2. Test functions                               test-functions-details

test_alloc_fail({id}{countdown}{repeat})            test_alloc_fail()
                This is for testing: If the memory allocation with {id} is
                called, then decrement {countdown}, and when it reaches zero
                let memory allocation fail {repeat} times.  When {repeat} is
                smaller than one it fails one time.

                Can also be used as a method:
                        GetAllocId()->test_alloc_fail()

                Return type: Number


test_autochdir()                                        test_autochdir()
                Set a flag to enable the effect of 'autochdir' before Vim
                startup has finished.

                Return type: Number


test_feedinput({string})                                test_feedinput()
                Characters in {string} are queued for processing as if they
                were typed by the user. This uses a low level input buffer.
                This function works only when with +unix or GUI is running.

                Can also be used as a method:
                        GetText()->test_feedinput()

                Return type: Number


test_garbagecollect_now()                        test_garbagecollect_now()
                Like garbagecollect(), but executed right away.  This must
                only be called directly to avoid any structure to exist
                internally, and v:testing must have been set before calling
                any function.   E1142
                This will not work when called from a :def function, because
                variables on the stack will be freed.

                Return type: Number

test_garbagecollect_soon()                       test_garbagecollect_soon()
                Set the flag to call the garbagecollector as if in the main
                loop.  Only to be used in tests.

                Return type: Number

test_getvalue({name})                                   test_getvalue()
                Get the value of an internal variable.  These values for
                {name} are supported:
                        need_fileinfo

                Can also be used as a method:
                        GetName()->test_getvalue()

                Return type: Number

                                                test_gui_event()
test_gui_event({event}{args})
                Generate a GUI {event} with arguments {args} for testing Vim
                functionality. This function works only when the GUI is
                running.

                {event} is a String and the supported values are:
                    "dropfiles" drop one or more files in a window.
                    "findrepl"  search and replace text.
                    "mouse"     mouse button click event.
                    "scrollbar" move or drag the scrollbar.
                    "key"       send a low-level keyboard event.
                    "tabline"   select a tab page by mouse click.
                    "tabmenu"   select a tabline menu entry.

                {args} is a Dict and contains the arguments for the event.

                "dropfiles":
                  Drop one or more files in a specified window.  The supported
                  items in {args} are:
                    files:      List of file names
                    row:        window row number
                    col:        window column number
                    modifiers:  key modifiers. The supported values are:
                                    0x4 Shift
                                    0x8 Alt
                                   0x10 Ctrl
                  The files are added to the argument-list and the first
                  file in {files} is edited in the window.  See drag-n-drop
                  for more information.  This event works only when the
                  drop_file feature is present.

                "findrepl":
                  {only available when the GUI has a find/replace dialog}
                  Perform a search and replace of text.  The supported items
                  in {args} are:
                    find_text:  string to find.
                    repl_text:  replacement string.
                    flags:      flags controlling the find/replace. Supported
                                values are:
                                    1   search next string (find dialog)
                                    2   search next string (replace dialog)
                                    3   replace string once
                                    4   replace all matches
                                    8   match whole words only
                                   16   match case
                    forward:    set to 1 for forward search.

                "mouse":
                  Inject either a mouse button click, or a mouse move, event.
                  The supported items in {args} are:
                    button:     mouse button.  The supported values are:
                                    0   left mouse button
                                    1   middle mouse button
                                    2   right mouse button
                                    3   mouse button release
                                    4   scroll wheel down
                                    5   scroll wheel up
                                    6   scroll wheel left
                                    7   scroll wheel right
                    row:        mouse click row number.  The first row of the
                                Vim window is 1 and the last row is 'lines'.
                    col:        mouse click column number.  The maximum value
                                of {col} is 'columns'.
                    multiclick: set to 1 to inject a multiclick mouse event.
                    modifiers:  key modifiers.  The supported values are:
                                    4   shift is pressed
                                    8   alt is pressed
                                   16   ctrl is pressed
                    move:       Optional; if used and TRUE then a mouse move
                                event can be generated.
                                Only {args} row: and col: are used and
                                required; they are interpreted as pixels or
                                screen cells, depending on "cell".
                                Only results in an event when 'mousemoveevent'
                                is set or a popup uses mouse move events.
                    cell:       Optional: when present and TRUE then "move"
                                uses screen cells instead of pixel positions

                "scrollbar":
                  Set or drag the left, right or horizontal scrollbar.  Only
                  works when the scrollbar actually exists.  The supported
                  items in {args} are:
                    which:      Selects the scrollbar. The supported values
                                are:
                                    left  Left scrollbar of the current window
                                    right Right scrollbar of the current window
                                    hor   Horizontal scrollbar
                    value:      Amount to scroll.  For the vertical scrollbars
                                the value can be between 0 to the line-count
                                of the buffer minus one.  For the horizontal
                                scrollbar the value can be between 1 and the
                                maximum line length, assuming 'wrap' is not
                                set.
                    dragging:   1 to drag the scrollbar and 0 to click in the
                                scrollbar.

                "key":
                  Send a low-level keyboard event (e.g. key-up or down).
                  Currently only supported on MS-Windows.
                  The supported items in {args} are:
                    event:      The supported string values are:
                                    keyup   generate a keyup event
                                    keydown generate a keydown event
                    keycode:    Keycode to use for a keyup or a keydown event.
                                                                E1291

                "tabline":
                  Inject a mouse click event on the tabline to select a
                  tabpage. The supported items in {args} are:
                    tabnr:      tab page number

                "tabmenu":
                  Inject an event to select a tabline menu entry. The
                  supported items in {args} are:
                    tabnr:      tab page number
                    item:       tab page menu item number. 1 for the first
                                menu item, 2 for the second item and so on.

                After injecting the GUI events you probably should call
                feedkeys() to have them processed, e.g.:
                        call feedkeys("y", 'Lx!')

                Returns TRUE if the event is successfully added, FALSE if
                there is a failure.

                Can also be used as a method:
                        GetEvent()->test_gui_event({args})

                Return type: vim9-boolean

test_ignore_error({expr})                        test_ignore_error()
                Ignore any error containing {expr}.  A normal message is given
                instead.
                This is only meant to be used in tests, where catching the
                error with try/catch cannot be used (because it skips over
                following code).
                {expr} is used literally, not as a pattern.
                When the {expr} is the string "RESET" then the list of ignored
                errors is made empty.

                Can also be used as a method:
                        GetErrorText()->test_ignore_error()

                Return type: Number

test_mswin_event({event}{args})               test_mswin_event()
                Generate a low-level MS-Windows {event} with arguments {args}
                for testing Vim functionality.  It works for MS-Windows GUI
                and for the console.

                {event} is a String and the supported values are:
                    "mouse"     mouse event.
                    "key"       keyboard event.
                    "set_keycode_trans_strategy"
                                Change the key translation method.

                "mouse":
                  Inject either a mouse button click, or a mouse move, event.
                  The supported items in {args} are:
                    button:     mouse button.  The supported values are:
                                    0   right mouse button
                                    1   middle mouse button
                                    2   left mouse button
                                    3   mouse button release
                                    4   scroll wheel down
                                    5   scroll wheel up
                                    6   scroll wheel left
                                    7   scroll wheel right
                    row:        mouse click row number.  The first row of the
                                Vim window is 1 and the last row is 'lines'.
                    col:        mouse click column number.  The maximum value
                                of {col} is 'columns'.
                                Note: row and col are always interpreted as
                                screen cells for the console application.
                                But, they may be interpreted as pixels
                                for the GUI, depending on "cell".
                    multiclick: set to 1 to inject a double-click mouse event.
                    modifiers:  key modifiers.  The supported values are:
                                    4   shift is pressed
                                    8   alt is pressed
                                   16   ctrl is pressed
                    move:       Optional; if used and TRUE then a mouse move
                                event can be generated.
                                Only {args} row: and col: are used and
                                required.
                                Only results in an event when 'mousemoveevent'
                                is set or a popup uses mouse move events.
                    cell:       Optional for the GUI: when present and TRUE
                                then "move" uses screen cells instead of pixel
                                positions.  Not used by the console.

                "key":
                  Send a low-level keyboard event (e.g. keyup or keydown).
                  The supported items in {args} are:
                    event:      The supported string values are:
                                    keyup   generate a keyup event
                                    keydown generate a keydown event
                    keycode:    Keycode to use for a keyup or a keydown event.
                    modifiers:  Optional; key modifiers.
                                The supported values are:
                                    2   shift is pressed
                                    4   ctrl is pressed
                                    8   alt is pressed
                                Note: These values are different from the
                                mouse modifiers.
                    execute:    Optional. Similar to feedkeys() mode x.
                                When this is included and set to true
                                (non-zero) then Vim will process any buffered
                                unprocessed key events.  All other {args}
                                items are optional when this is set and true.

                "set_keycode_trans_strategy":
                  w32-experimental-keycode-trans-strategy
                  Switch the keycode translation method. The supported methods
                  are:
                    experimental:   The method used after Patch v8.2.4807
                                    using ToUnicode() Win API call.
                    classic:        The method used pre Patch v8.2.4807
                                    using the TranslateMessage() Win API call.

                Returns TRUE if the event is successfully added or executed,
                FALSE if there is a failure.

                Can also be used as a method:
                        GetEvent()->test_mswin_event({args})

                Return type: vim9-boolean

test_null_blob()                                        test_null_blob()
                Return a Blob that is null. Only useful for testing.

                Return type: Blob

test_null_channel()                                     test_null_channel()
                Return a Channel that is null. Only useful for testing.
                {only available when compiled with the +channel feature}

                Return type: Channel

test_null_dict()                                        test_null_dict()
                Return a Dict that is null. Only useful for testing.

                Return type: dict<any>

test_null_function()                                    test_null_function()
                Return a Funcref that is null. Only useful for testing.

                Return type: func(...): unknown

test_null_job()                                         test_null_job()
                Return a Job that is null. Only useful for testing.
                {only available when compiled with the +job feature}

                Return type: job

test_null_list()                                        test_null_list()
                Return a List that is null. Only useful for testing.

                Return type: list<any>

test_null_partial()                                     test_null_partial()
                Return a Partial that is null. Only useful for testing.

                Return type: func(...): unknown

test_null_string()                                      test_null_string()
                Return a String that is null. Only useful for testing.

                Return type: String

test_option_not_set({name})                             test_option_not_set()
                Reset the flag that indicates option {name} was set.  Thus it
                looks like it still has the default value. Use like this:
                        set ambiwidth=double
                        call test_option_not_set('ambiwidth')
                Now the 'ambiwidth' option behaves like it was never changed,
                even though the value is "double".
                Only to be used for testing!

                Can also be used as a method:
                        GetOptionName()->test_option_not_set()

                Return type: Number

test_override({name}{val})                            test_override()
                Overrides certain parts of Vim's internal processing to be able
                to run tests. Only to be used for testing Vim!
                The override is enabled when {val} is non-zero and removed
                when {val} is zero.
                Current supported values for {name} are:

                {name}       effect when {val} is non-zero
                alloc_lines  make a copy of every buffer line into allocated
                             memory, so that memory access errors can be found
                             by valgrind.
                autoload     import autoload will load the script right
                             away, not postponed until an item is used.
                char_avail   disable the char_avail() function.
                defcompile   all the :def functions in a sourced script are
                             compiled when defined.  This is similar to using
                             the :defcompile command in a script.
                nfa_fail     makes the NFA regexp engine fail to force a
                             fallback to the old engine.
                no_query_mouse  do not query the mouse position for "dec"
                                terminals.
                no_wait_return  set the "no_wait_return" flag.  Not restored
                                with "ALL".
                redraw       disable the redrawing() function.
                redraw_flag  ignore the RedrawingDisabled flag.
                starting     reset the "starting" variable, see below.
                term_props   reset all terminal properties when the version
                             string is detected.
                ui_delay     time in msec to use in ui_delay(); overrules a
                             wait time of up to 3 seconds for messages.
                unreachable  no error for code after :throw and :return.
                uptime       overrules sysinfo.uptime.
                vterm_title  setting the window title by a job running in a
                             terminal window.
                ALL          clear all overrides, except alloc_lines ({val} is
                             not used).

                "starting" is to be used when a test should behave like
                startup was done.  Since the tests are run by sourcing a
                script the "starting" variable is non-zero. This is usually a
                good thing (tests run faster), but sometimes this changes
                behavior in a way that the test doesn't work properly.
                When using:
                        call test_override('starting', 1)
                The value of "starting" is saved.  It is restored by:
                        call test_override('starting', 0)

                To make sure the flag is reset later using :defer can be
                useful:
                        call test_override('unreachable', 1)
                        defer call test_override('unreachable', 0)

                Can also be used as a method:
                        GetOverrideVal()-> test_override('starting')

                Return type: Number

test_refcount({expr})                                   test_refcount()
                Return the reference count of {expr}.  When {expr} is of a
                type that does not have a reference count, returns -1.  Only
                to be used for testing.

                Can also be used as a method:
                        GetVarname()->test_refcount()

                Return type: Number

test_setmouse({row}{col})                             test_setmouse()
                Set the mouse position to be used for the next mouse action.
                {row} and {col} are one based.
                For example:
                        call test_setmouse(4, 20)
                        call feedkeys("\<LeftMouse>", "xt")

                Return type: Number


test_settime({expr})                                    test_settime()
                Set the time Vim uses internally.  Currently only used for
                timestamps in the history, as they are used in viminfo, and
                for undo.
                Using a value of 1 makes Vim not sleep after a warning or
                error message.
                {expr} must evaluate to a number.  When the value is zero the
                normal behavior is restored.

                Can also be used as a method:
                        GetTime()->test_settime()

                Return type: Number

test_srand_seed([{seed}])                               test_srand_seed()
                When {seed} is given this sets the seed value used by
                srand().  When omitted the test seed is removed.

                Return type: Number

test_unknown()                                          test_unknown()
                Return a value with unknown type. Only useful for testing.

                Return type: unknown

test_void()                                             test_void()
                Return a value with void type. Only useful for testing.

                Return type: void

==============================================================================
3. Assert functions                             assert-functions-details


assert_beeps({cmd})                                     assert_beeps()
                Run {cmd} and add an error message to v:errors if it does
                NOT produce a beep or visual bell.
                Also see assert_fails()assert_nobeep() and
                assert-return.

                Can also be used as a method:
                        GetCmd()->assert_beeps()

                Return type: Number

                                                        assert_equal()
assert_equal({expected}{actual} [, {msg}])
                When {expected} and {actual} are not equal an error message is
                added to v:errors and 1 is returned.  Otherwise zero is
                returned. assert-return
                The error is in the form "Expected {expected} but got
                {actual}".  When {msg} is present it is prefixed to that,
                along with the location of the assert when run from a script.

                There is no automatic conversion, the String "4" is different
                from the Number 4.  And the number 4 is different from the
                Float 4.0.  The value of 'ignorecase' is not used here, case
                always matters.
                Example:
                        call assert_equal('foo', 'bar', 'baz')
                Will add the following to v:errors:
                        test.vim line 12: baz: Expected 'foo' but got 'bar'

                Can also be used as a method, the base is passed as the
                second argument:
                        mylist->assert_equal([1, 2, 3])

                Return type: Number

                                                        assert_equalfile()
assert_equalfile({fname-one}{fname-two} [, {msg}])
                When the files {fname-one} and {fname-two} do not contain
                exactly the same text an error message is added to v:errors.
                Also see assert-return.
                When {fname-one} or {fname-two} does not exist the error will
                mention that.
                Mainly useful with terminal-diff.

                Can also be used as a method:
                        GetLog()->assert_equalfile('expected.log')

                Return type: Number

assert_exception({error} [, {msg}])                     assert_exception()
                When v:exception does not contain the string {error} an error
                message is added to v:errors.  Also see assert-return.
                This can be used to assert that a command throws an exception.
                Using the error number, followed by a colon, avoids problems
                with translations:
                        try
                          commandthatfails
                          call assert_false(1, 'command should have failed')
                        catch
                          call assert_exception('E492:')
                        endtry

                Return type: Number

                                                        assert_fails()
assert_fails({cmd} [, {error} [, {msg} [, {lnum} [, {context}]]]])
                Run {cmd} and add an error message to v:errors if it does
                NOT produce an error or when {error} is not found in the
                error message.  Also see assert-return.
                                                        E856
                When {error} is a string it must be found literally in the
                first reported error. Most often this will be the error code,
                including the colon, e.g. "E123:".
                        call assert_fails('bad cmd', 'E987:')

                When {error} is a List with one or two strings, these are
                used as patterns.  The first pattern is matched against the
                first reported error:
                        call assert_fails('cmd', ['E987:.*expected bool'])
                The second pattern, if present, is matched against the last
                reported error.
                If there is only one error then both patterns must match. This
                can be used to check that there is only one error.
                To only match the last error use an empty string for the first
                error:
                        call assert_fails('cmd', ['', 'E987:'])

                If {msg} is empty then it is not used.  Do this to get the
                default message when passing the {lnum} argument.
                                                        E1115
                When {lnum} is present and not negative, and the {error}
                argument is present and matches, then this is compared with
                the line number at which the error was reported. That can be
                the line number in a function or in a script.
                                                        E1116
                When {context} is present it is used as a pattern and matched
                against the context (script name or function name) where
                {lnum} is located in.

                Note that beeping is not considered an error, and some failing
                commands only beep.  Use assert_beeps() for those.

                Can also be used as a method:
                        GetCmd()->assert_fails('E99:')

                Return type: Number

assert_false({actual} [, {msg}])                        assert_false()
                When {actual} is not false an error message is added to
                v:errors, like with assert_equal().
                The error is in the form "Expected False but got {actual}".
                When {msg} is present it is prefixed to that, along with the
                location of the assert when run from a script.
                Also see assert-return.

                A value is false when it is zero. When {actual} is not a
                number the assert fails.

                Can also be used as a method:
                        GetResult()->assert_false()

                Return type: Number

assert_inrange({lower}{upper}{actual} [, {msg}])     assert_inrange()
                This asserts number and Float values.  When {actual}  is lower
                than {lower} or higher than {upper} an error message is added
                to v:errors.  Also see assert-return.
                The error is in the form "Expected range {lower} - {upper},
                but got {actual}".  When {msg} is present it is prefixed to
                that.

                Return type: Number

                                                                assert_match()
assert_match({pattern}{actual} [, {msg}])
                When {pattern} does not match {actual} an error message is
                added to v:errors.  Also see assert-return.
                The error is in the form "Pattern {pattern} does not match
                {actual}".  When {msg} is present it is prefixed to that,
                along with the location of the assert when run from a script.

                {pattern} is used as with =~: The matching is always done
                like 'magic' was set and 'cpoptions' is empty, no matter what
                the actual value of 'magic' or 'cpoptions' is.

                {actual} is used as a string, automatic conversion applies.
                Use "^" and "$" to match with the start and end of the text.
                Use both to match the whole text.

                Example:
                        call assert_match('^f.*o$', 'foobar')
                Will result in a string to be added to v:errors:
                        test.vim line 12: Pattern '^f.*o$' does not match 'foobar'

                Can also be used as a method:
                        getFile()->assert_match('foo.*')

                Return type: Number

assert_nobeep({cmd})                                    assert_nobeep()
                Run {cmd} and add an error message to v:errors if it
                produces a beep or visual bell.
                Also see assert_beeps().

                Can also be used as a method:
                        GetCmd()->assert_nobeep()

                Return type: Number

                                                        assert_notequal()
assert_notequal({expected}{actual} [, {msg}])
                The opposite of assert_equal(): add an error message to
                v:errors when {expected} and {actual} are equal.
                Also see assert-return.

                Can also be used as a method:
                        mylist->assert_notequal([1, 2, 3])

                Return type: Number

                                                        assert_notmatch()
assert_notmatch({pattern}{actual} [, {msg}])
                The opposite of assert_match(): add an error message to
                v:errors when {pattern} matches {actual}.
                Also see assert-return.

                Can also be used as a method:
                        getFile()->assert_notmatch('bar.*')

                Return type: Number

assert_report({msg})                                    assert_report()
                Report a test failure directly, using String {msg}.
                Always returns one.

                Can also be used as a method:
                        GetMessage()->assert_report()

                Return type: Number


assert_true({actual} [, {msg}])                         assert_true()
                When {actual} is not true an error message is added to
                v:errors, like with assert_equal().
                Also see assert-return.
                A value is TRUE when it is a non-zero number.  When {actual}
                is not a number the assert fails.
                When {msg} is given it is prefixed to the default message,
                along with the location of the assert when run from a script.

                Can also be used as a method:
                        GetResult()->assert_true()

                Return type: Number


 vim:tw=78:ts=8:noet:ft=help:norl: