NAME
Covered - Verilog Code Coverage Analyzer
SYNTAX
covered [global_options] score [options]
covered [global_options] merge [options] existing_database
database_to_merge+
covered [global_options] report [options] database_file
covered [global_options] rank [options] database_to_rank
database_to_rank+
covered [global_options] exclude [options] exclusion_id+ database_file
DESCRIPTION
Covered is a Verilog code coverage analysis tool that can be useful for
determining how well a diagnostic test suite is covering the design
under test. Covered reads in the Verilog design files and a VCD or LXT2
formatted dumpfile from a diagnostic run and generates a database file
called a Coverage Description Database (CDD) file, using the score
command. Covered’s score command can alternatively be used to generate
a CDD file and a Verilog module for using Covered as a VPI module in a
testbench which can obtain coverage information in parallel with
simulation (see USING COVERED AS A VPI MODULE). The resulting CDD file
can be merged with other CDD files from the same design to create
accummulated coverage, using the merge command. Once a CDD file is
created, the user can use Covered to generate various human-readable
coverage reports in an ASCII format or use Covered’s GUI to
interactively look at coverage results, using the report command. If
uncovered coverage points are found that the user wants to exclude from
coverage, this can be handled with either the command-line exclude
command or within the GUI. When multiple CDD files are created from
the same design, the user may obtain a coverage ranking of those CDD
files to determine an ideal order for regression testing as well as
understand which CDD files can be excluded from regressions due to
their inability to hit new coverage points. Additionally, as part of
Covered’s score command, race condition possibilities are found in the
design files and can be either ignored, flagged as warnings or flagged
as errors. By specifying race conditions as errors, Covered can also
be used as a race condition checker.
GLOBAL OPTIONS
These options are placed immediately after the keyword covered in the
command-line. They can be used for any command (with the exception of
-v and -h) and have the same effect in each case.
-B Obfuscate. Obfuscates all design-sensitive names before
outputting in user-readable format. This option is useful when
sharing output with the developers of Covered for debugging
purposes.
-D Debug. Display information helpful for debugging tool problems.
Note: This option is now only available when covered is built
with the --enable-debug configuration option.
-h Help. Display this usage information.
-P [filename]
Profiling mode. Turns on internal source code profiler that
will produce a profiling report of the run command to either the
specified filename or, if no filename is present, to a file
called covered.prof. This option is only available if the
--enable-profiling configuration option was specified when
Covered was built.
-Q Quiet mode. Causes all output to be suppressed.
-T Terse mode. Causes all output to be suppressed with the
exception of warning messages and the Covered header
information.
-v Version. Display current Covered version.
COMMANDS
score Parses Verilog files and VCD/LXT2 dumpfiles to create database
file used for merging and reporting.
merge Merges two or more database files into one.
report Generates human-readable coverage reports from database file or
starts the coverage report GUI.
rank Generates a report that specifies an ideal order to run
regressions and specifies CDD files that do not add new coverage
information (and can, therefore, be excluded from regressions,
if desired).
exclude
Allows one or more coverage points (identified with exclusion
IDs in a report generated with the -x option) to have their
exclusion property toggled (include to exclude or exclude to
include) and, if the exclusion property is set to exclude,
optionally allow a reason for the exclusion to be associated
with it and stored in the specified CDD file.
SCORE COMMAND
The following options are valid for the score command:
-A ovl Causes OVL assertions to be used for assertion coverage. This
flag must be given to the score command if assertion coverage
metrics are needed in the report command.
-cdd database
Same as the -o option. Useful when CDD file being scored is an
input to the score command.
-cli [filename]
Causes the command-line debugger to be used during VCD/LXT2
dumpfile scoring. If filename is specified, this file contains
information saved in a previous call to savehist on the CLI and
causes the history contained in this file to be replayed prior
to the CLI command prompt. If filename is not specified, the
CLI prompt will be immediately available at the start of
simulation. This option is only available when Covered is
configured with the --enable-debug option.
-conservative
If this option is specified, any logic blocks that contain code
that could cause coverage discrepancies leading to potentially
inaccurate coverage results are removed from coverage
consideration. See User’s Guide for more information on what
type of code can lead to coverage inaccuracies.
-dumpvars [filename]
If this option is specified without the -vcd or -lxt options,
the design is parsed, a CDD file is created and a top-level
Verilog module file named filename (if this value is specified)
or "covered_dump.v" (if filename is not specified) is created.
This file is used in the compilation of the simulator to create
a dumpfile that is optimized for obtaining coverage for the
specified CDD file. If either the -vcd or -lxt options are
specified, this option has no effect. See the User’s Guide for
more information on how to use this option.
-D define_name
Defines the specified name to 1.
-D define_name=value
Defines the specified name to the specified value.
-e block_name
Name of module, task, function or named begin/end block to not
score. Causes all subblocks in the Verilog tree under this
block to also not be scored.
-ea Excludes all always blocks from being considered for coverage.
-ec Excludes all continuous assignments from being considered for
coverage.
-ei Excludes all initial blocks from being considered for coverage.
-ef Excludes all final blocks from being considered for coverage.
-ep [name]
Excludes all code found between ’// coverage off’ and ’//
coverage on’ pragmas embedded within the design. If name is
specified, the keyword in the pragma will be changed to that
name instead of the default "coverage", allowing the user to
change the look of the pragma if it conflicts with any other
tools.
-F module_name=[in_expr,]out_expr
Indicates to the parser where to find the FSM located in module
module_name which has an input state expression called in_expr
and output state expression called out_expr. If in_expr is not
specified, out_expr is used as both the input and output state
expression.
-f filename
Name of file containing additional arguments to parse.
-g [module_name=](1|2|3)
Allows the user to limit the parser to a specific generation of
the Verilog standard for a specific module or the entire design,
where 1=Verilog-1995, 2=Verilog-2001, 3=SystemVerilog. If
module_name= is specified, the parser uses the specified Verilog
standard for that module only, allowing the rest of the design
to use the global standard. By default, the global standard is
set to the most recent version (3). This option can be
specified more than once for a given call to the score command;
however, if the -g option specifies more than global value
(i.e., without the module= prefix), only the last option value
will be used.
-h Displays this help information.
-I directory
Directory to find included Verilog files.
-i instance_name
Verilog hierarchical reference to the module that is at the top
of the tree to be scored. This option is necessary if module to
verify coverage is not the top-level module in the design. If
not specified, -t value is used.
-lxt filename
Name of LXT2 dumpfile to score design with. If this or the -vcd
option is not used, Covered will only create an initial CDD file
from the design and will not attempt to score the design.
-m message
Allows the user to specify information about this CDD file.
This information can be anything (messages with whitespace
should be surrounded by double-quotation marks), but may include
something about the simulation arguments to more easily link the
CDD file to its simulation for purposes of recreating the CDD
file.
-o database
Name of database to write coverage information to. If not
specified, the output database filename will be "cov.cdd".
-p filename
Overrides default filename used to store intermediate
preprocessor output.
-P parameter_scope=value
Performs a defparam on the specified parameter with value.
-rS When race condition checks are violated, the offending logic
blocks are removed from coverage consideration and all output is
suppressed regarding the race condition violation. See user
documentation for more information about race condition checking
usage.
-rW When race condition checks are violated, the offending logic
blocks are removed from coverage consideration and the race
condition violation is output. This is the default behavior for
race condition handling. See user documentation for more
information about race condition checking usage.
-rE When race condition checks are violated, the reason is output
and scoring ends immediately. See user documentation for more
information about race condition checking usage.
-rI[=module name]
If module name is not specified, race condition checking is
skipped altogether for the entire design. If module name is
specified, race condition checking is skipped for the specified
module. See user documentation for more information about race
condition checking usage.
-rP[=name])
Uses embedded pragmas for ignoring certain code from race
condition checking consideration (if name is specified it is
used as the pragma keyword). See user documentation for more
information about race condition checking usage.
-S Outputs simulation statistics after simulation has completed.
This information is currently only useful for the developers of
Covered.
-t top-level module
Specifies the module name of the top-most module that will be
measured. Note that this module does not need to be the
top-most module in the simulator. This field is required for
all calls to the score command.
-top_ts timescale
This option is only valid when the -vpi or -dumpvars options
have been specified. This option allows the user to specify a
timescale for the generated Verilog module created with the
-vpi/-dumpvars option. If this option is not specified, no
timescale will be created for the generated module. The value
of timescale is specified as follows:
(1|10|100)(s|ms|us|ns|ps|fs)/(1|10|100)(s|ms|us|ns|ps|fs)
If whitespace is needed between the various values, place the
entire contents of timescale in double quotes.
-ts number
When scoring occurs, this option allows the user to see how far
the simulator has progressed by outputting the current timestep
to standard output. The value of number specifies how many
timesteps are allowed to be simulated before outputting the
current timestep (results in less calls to output stream).
-T (min|typ|max)
Specifies which value to use when encountering a delay
expression in the form: min:typ:max. If this option is not
specified, ’typ’ select is used by default.
-v filename
Name of specific Verilog file to score.
-vcd filename
Name of VCD dumpfile to score design with. If this or the -lxt
option is not used, Covered will only create an initial CDD file
from the design and will not attempt to score the design.
-vpi [filename]
If this option is specified without the -vcd or -lxt options,
the design is parsed, a CDD file is created and a top-level
Verilog module file named filename (if this value is specified)
or "covered_vpi.v" (if filename is not specified) is created
along with a PLI table file called filename.ta b or
"covered_vpi.v.ta b". Both of these files are used in the
compilation of the simulator to use Covered as a VPI module. If
either the -vcd or -lxt options are specified, this option has
no effect.
-Wignore
Suppress the output of warnings during code parsing and
simulation.
-y directory
Directory to find unspecified Verilog files.
+libext+.extension[+.extension]*+
Extensions of Verilog files to allow in scoring.
MERGE COMMAND
The following options are valid for the merge command:
-d filename
Directory to search for CDD files to include. This option is
used in conjunction with the -ext option which specifies the
file extension to use for determining which files in the
directory are CDD files.
-er (first|last|all|new|old)
Specifies how to handle exclusion reason resolution. If two or
more CDD files being merged have exclusion reasons specified for
the same coverage point, the exclusion reason needs to be
resolved (unless it is the same string value). If this option
is not specified and a conflict is found, Covered will
interactively request input for each exclusion as to how to
handle it. If this option is specified, it tells Covered how to
handle all exclusion reason conflicts. The values are as
follows:.br
first - CDD file that contained the first exclusion reason is
used.
last - CDD file that contained the last exclusion reason is
used.
all - All exclusion reasons are used (concatenated).
new - Use the newest exclusion reason specified.
old - Use the oldest exclusion reason specified.
-ext extension
Used in conjunction with the -d option. If no -ext options are
specified on the command-line, the default value of ’.cdd’ is
used. Note that a period (.) should be specified.
-f filename
Name of file containing additional arguments to parse.
-h Displays this help information.
-m message
Allows the user to specify information about this CDD file.
This information can be anything (messages with whitespace
should be surrounded by double-quotation marks).
-o filename
File to output new database to. If this argument is not
specified, the existing_database is used as the output database
name.
REPORT COMMAND
The following options are valid with the report command:
-b If combinational logic verbose output is reported and the
expression is a vector operation, this option outputs the
coverage information on a bitwise basis.
-c If -v is specified, displays covered metrics only. Default is
to display uncovered information only.
-d (s|d|v)
Level of detail to provide in coverage report information (s =
summary, d = detailed, v = verbose). Default is summary.
-e Outputs all excluded coverage points to the report file along
with any specified exclusion reasons if the -d d or -d v options
are specified.
-f filename
Name of file containing additional arguments to parse.
-h Displays this help information.
-i Provides coverage information for instances instead of module.
-m [l][t][c][f][r][a][m]
Type(s) of metrics to report. l=line, t=toggle, c=combinational
logic, f=FSM state and state transition, r=race conditions,
a=assertion, m=memory. Default is ltcf.
-o filename
File to output report information to. Default is standard
output.
-s Suppresses modules/instances that contain no coverage
information from being output to the report. Used to help
eliminate potentially meaningless information from the report.
-v Deprecated. Replaced by ’-d d’ or ’-d v’.
-view Starts the GUI interface for interactive coverage reporting.
-w (number)
Specifies the maximum line width (in characters) that can be
used to output Verilog information. If this option is not
specified, all Verilog code in the report will retain the same
formatting as was specified in the original Verilog code. If
this option is specified, Verilog code will be formatted to use
as much of the current line as possible, wrapping text when the
line reaches the maximum line width. The default maximum line
width is 115 characters (this value is used if no number is
specified with the -w option). If a number is specified with
the -w option, this value is used for the maximum line width.
-x Outputs the exclusion IDs of all uncovered and excluded coverage
points within parenthesis before the associated verbose output
of the coverage point. The exclusion IDs can be used to
exclude/include coverage points via the exclude command.
RANK COMMAND
The following options are valid with the rank command:
-d filename
Directory to search for CDD files to include. This option is
used in conjunction with the -ext option which specifies the
file extension to use for determining which files in the
directory are CDD files.
-depth number
Specifies the minimum number of needed CDD files to hit each
coverage point. The value of number should be greater than
zero. Default is 1.
-ext extension
Used in conjunction with the -d option. If no -ext options are
specified on the command-line, the default value of ’.cdd’ is
used. Note that a period (.) should be specified.
-f filename
Name of file containing additional arguments to parse.
-h Displays help information for the rank command.
-names-only
If specified, outputs only the needed CDD filenames that need to
be run in the order they need to be run. If this option is not
set, a report-style output is provided with additional
information. This option is meant to be useful in scripts that
only want CDD filenames to run as output.
-o filename
Name of file to output ranking information to. Default is
standard output.
-required-cdd filename
Name of CDD that should be considered a required CDD to rank
(i.e., it cannot be excluded for any reason).
-required-list filename
Name of a file that contains a list of CDDs that should be
considered required CDDs to rank. The filenames should be
separated by whitespace or newline characters within the file.
-v Causes verbose output to be displayed when the rank command is
run. It outputs diagnostic information about each of the
different phases of the ranking algorithm including run-time,
number of CDD files included/excluded and number of coverage
points hit by ranked CDDs during each phase. This information
is meant to be useful for those interested in the ranking
algorithm and its performance.
-weight-assert number
Specifies a relative weighting for assertion coverage used to
rank non-unique coverage points. The value of number is
relative to the values used in the -weight-toggle,
-weight-memory, -weight-comb, -weight-fsm and -weight-line rank
command options.
-weight-comb number
Specifies a relative weighting for combinational logic coverage
used to rank non-unique coverage points. The value of number is
relative to the values used in the -weight-toggle,
-weight-memory, -weight-assert, -weight-fsm and -weight-line
rank command options.
-weight-fsm number
Specifies a relative weighting for FSM state and state
transition coverage used to rank non-unique coverage points.
The value of number is relative to the values used in the
-weight-toggle, -weight-memory, -weight-comb, -weight-assert and
-weight-line rank command options.
-weight-line number
Specifies a relative weighting for line coverage used to rank
non-unique coverage points. The value of number is relative to
the values used in the -weight-toggle, -weight-memory,
-weight-comb, -weight-fsm and -weight-assert rank command
options.
-weight-memory number
Specifies a relative weighting for memory coverage used to rank
non-unique coverage points. The value of number is relative to
the values used in the -weight-toggle, -weight-line,
-weight-comb, -weight-fsm and -weight-assert rank command
options.
-weight-toggle number
Specifies a relative weighting for toggle coverage used to rank
non-unique coverage points. The value of number is relative to
the values used in the -weight-memory, -weight-line,
-weight-comb, -weight-fsm and -weight-assert rank command
options.
EXCLUDE COMMAND
The following options are valid with the exclude command:
-f filename
Specifies the name of a file that contains more options to the
exclude command. This option may be specified as many times as
necessary for a single call to the exclude command.
-h Generates usage information for the exclude command.
-m Allows for an exclusion message to be associated with any
coverage points going from the included state to the excluded
state. For each coverage point that meets this requirement, the
user will be prompted to input a reason. The reason may be any
length and any number of lines; however, all formatting
characters (i.e., newlines, tabs, extra spaces, etc.) will be
removed and replaced with a single space when it is later
displayed. To end the input of a message, hit a return, enter a
single period (.) character and hit return again. The final
period character will not be part of the exclusion message.
-p Causes all specified coverage points to print their current
exclusion status and exclusion reason (if one exists for the
excluded coverage point) to standard output. If this option is
specified, the -m option will be ignored.
USING COVERED AS A VPI MODULE
In addition to using Covered’s score command to parse a VCD or LXT2
file to abstract coverage information, Covered may also be used as a
VPI module within a simulator to extract this information. The
advantages to using Covered as a VPI over a dumpfile reader include the
following. First, VCD files can be extrememly large, especially for
long simulations, using up valuable disk space. Second, if you are
using a simulator that dumps files in a different format than VCD or
LXT2 and you want to convert these dump file types to one of these
versions, the cost of disk space and time can make creating dumpfiles
that Covered requires undesirable. Additionally, though using Covered
as a VPI module will slow down your simulation speed, it is most likely
that the total time spent simulating your design and scoring the design
in one step will be shorter than doing so in two steps. As a result,
Covered’s configure utility can generate VPI-ready libraries for the
following free and commercial simulators (Icarus Verilog, CVER and
VCS).
To automatically build the VPI-ready library files when generating
Covered from source, simply specify one or more of the following when
running the "configure" utility in the base Covered directory:
--with-iv=<Icarus Verilog install path>, --with-vcs=<VCS include path>,
--with-cver=<CVER include path>. After Covered has been configured,
simply type ’make’ and ’make install’. This will install the VPI-ready
library files in the installation libexec directory (by default this
path will be /usr/local/libexec).
Before you are ready to compile the design, you must first create a CDD
file, a top-level Verilog file, and a PLI table file (the last file is
only needed for the VCS compiler). This is done by specifying the -vpi
(filename) option to Covered’s score command. If no filename is
specified after -vpi, the files covered_vpi.v and covered_vpi.ta b will
be created along with the generated coverage file. Note that this step
only needs to be performed once unless the design files change. You
are now ready to compile the simulator.
If you are compiling an Icarus Verilog simulation, simply add ’-m
/usr/local/libexec/covered.vpi covered_vpi.v’ to the ’iverilog’
command-line. Once compilation is complete, run the generated
executable file as you normally would.
If you are compiling a CVER simulation, simply add
’+loadvpi=/usr/local/libexec/covered.cver.so:vpi_compat_bootstrap
covered_vpi.v’ to the ’cver’ command-line.
If you are compiling a VCS simulation, simply add ’+vpi -load
/usr/local/libexec/covered.vcs.so:covered_register covered_vpi.v’ to
the ’vcs’ command-line. Once compilation is complete, run the
generated executable file as you normally would.
If you are compiling a NC-Verilog simulation, switch to NC-Verilog’s
irun command to load the covered shared object: ’-loadvpi
/usr/local/libexec/covered.ncv.so:covered_register’ and enable all
access with ’-access +rwc’. You can hardcode the $covered_sim call into
your RTL or you can run it dynamically using the CLI, by adding the
-input input.tcl switch to irun. Where the input.tcl file looks like
the following and tb.dut is the coverage instance:
call -systf {$covered_sim} {"scored.cdd"} tb.dut
run
There are two plusargs that can be passed to the generated executable
when it is run that Covered will parse. The ’+covered_cdd=<filename>’
option will cause Covered to output the scored design contents to the
CDD file specified by <filename>. This allows multiple runs of the
simulator to generate several different CDD files without needed a
recompile to occur. The ’+covered_debug’ option will cause Covered to
dump a lot of excessive output about its internal run-time state during
simulation. This output will only be generated if Covered was
configured with the --enable-debug option. This plusarg option should
not be used by regular users as it is primarily intended to aid the
developers of Covered in debugging.
AUTHORS
Trevor Williams <phase1geo@gmail.com>
SEE ALSO
For more information on how to use the Covered code coverage tool,
please consult the User’s Guide included with this release at
/usr/local/share/covered/doc/html/index.html.