All About UVM Reporting

  

 What we have in Verilog and SystemVerilog for Reporting.

        Verilog and SystemVerilog have some system tasks for reporting and displaying messages. We all used $display, $error, $info, $monitor, $write, $fatal, $strobe and whatnot for different purpose like display data or messages, display errors, formatted messages and etc. Without going into detail, quickly recap these system tasks.

• $display: Printing a formatted and non-formatted message in the console. Automatically add a new line at end of the message.
• $info: Indicated assertion fail or not. It prints messages with additional information like time, path, line no and etc.
• $warning: It provides a run-time warning and is suppressed by a tool.
• $error: It provides a run-time error message.
• $fatal: It provides run-time fatal error and terminated simulation when it encounters.   
• $monitor: Keep tracing variables and printing them as and when changes occur.
• $strobe: Similar to $display, except message print at end of simulations. So it provides the last value or message at end of the simulation  

        But here a big question arises what are the controlling features?  Are they controlled or handle during simulation or runtime? So for answers of these question is "YES" and "NO" both. For "YES", we can develop our own containers and methods or macros and used them during the Test-Bench environment. But it is not global for all developers. It may deffer developer to developer. Countering this issue UVM has one unique approach. So let's dive into UVM reporting.

UVM Reporting:

        UVM reporting contains so many functionalities, methods, macros, and run-time commands to alter and change messages, their type and etc. UVM reporting is divided into three parts.

1. Verbosity: Provide "Filter level".  e.g. LOW, MID, HIGH, DEBUG, FULL.
2. Severity: Severity means "Importance". It filters the message types. e.g. Info, Warning, Error, Fatal.
3. Simulation handling behavior: Control simulation behavior during run time and provide more flexibility to run a simulation.

       Each of them independently controls and is specified as and when required.

        

UVM Reporting Marcos: 

        UVM reporting provides four macros based on that all reporting can be done.

        1)  `uvm_info: display simple messages and data. It same as $info and $display.

              Syntax: `uvm_info(Message ID, Message, Verbosity)

              Example: `uvm_info("string_id", "uvm_info test message", UVM_LOW)

        2)  `uvm_warning: Help to display run time warnings. it the same as $warning.

              Syntax: `uvm_warning(Message ID, Message)

              Example: `uvm_warning("string_id", "uvm_warning test message")

        3)  `uvm_error: Help to display run time errors. it the same as $error.

              Syntax: `uvm_error(Message ID, Message)

              Example: `uvm_error("string_id", "uvm_error test message")

        4)  `uvm_fatal: Help to display run time fatal. it the same as $fatal.

              Syntax: `uvm_fatal(Message ID, Message)

              Example: `uvm_fatal("string_id", "uvm_fatal test message")

        5) `uvm_*_contaxt: The `uvm_*_contaxt variant of the macros can be used when you want the                                                         message to appear as if emitted by another object.

             Syntax: `uvm_*_contaxt(Message ID, Message, contaxt)

             Example: `uvm_fatal_contaxt("string_id", "uvm_fatal test message", drv)

        If message is formatted or contain variable then used $sformatf("%s",string_name).

        Example: `uvm_info("string_id", $sformatf("%s",string_name), UVM_LOW)

Messages Verbosity Levels:

        Fundamentally, verbosity levels suggest how verbose messages are. If messages are only for debugging purposes then set as low or debug, if it is important for analysis or printing status of the process then used high or medium, if a message was very crucial like test pass or fail then used high verbosity level. There are different verbosity levels supported by UVM like UVM_NONE, UVM_LOW, UVM_MEDIUM, UVM_HIGH, UVM_FULL, and UVM_DEBUG. As per message type and its importance we can set any of this. Verbosity level and its associated values are given below. Based on that all messages filter out and printed.

        

Table-1: Verbosity level and associated values

        NOTE: 1) A message with UVM_NONE can not be printed.

                      2) Higher values, automatically enable lower verbosity messages.

                      3) `uvm_warning, `uvm_error, `uvm_fatal can't filter with verbosity.

        we can set verbosity level by using command-line argument and by using reporting method. Both have their own advantages and disadvantages. 

Verbosity level set from the command line:

        By use of the command line, the biggest advantage is we don't worry about individual components. It applies to all files and components. It means if we want to change verbosity and generate different log information with different verbosity levels then no need to recompile Test-bench. It will appear in the next simulation run.

+UVM_VERBOSITY=<VERBOSITY>

Example: +UVM_VERBOSITY=UVM_LOW

+uvm_set_verbosity=<Component/Hierarchy>,<Message_ID>,<Verbosity>,<Phase>

Example: +uvm_set_verbosity=uvm_test_top.pr,print_reporting,UVM_HIGH,run

Verbosity level set from the individual component:

        Verbosity level also set through reporting method which is pre-define in UVM library. It provides more flexibility towards components and it allows to set individual component verbosity levels. Also, we can set it in a hierarchy based.

set_report_verbosity_level(<VERBOSITY>);

Example: pr.set_report_verbosity_level(UVM_HIGH);

        If you want to apply verbosity level to component and its children then used set_report_verbosity_level_hier(<VERBOSITY>);

        Apart from that, the message ID-based verbosity level is also you can set by using set_report_id_verbosity(message_id, verbosity). For severity-based and message ID-based verbosity levels also you can set by using set_report_severity_id_verbosity(severity, message_id, verbosity).

Modified simulation action:

        In UVM, six types of actions were given which provides more flexibility towards uvm reporting severity. It helps to analyze run-time simulation through count errors, exit simulation if warning or errors beyond a set threshold, help in log files, etc.

Table-2: Simulation action and its description 

        For use, this simulation action in any test bench used below methods. For more details click here.

1. set_report_severity_action (uvm_severity severity, uvm_action action);
2. set_report_id_action (string message_ID, uvm_action action);
3. set_report_severity_id_action (uvm_severity severity, string message_ID, uvm_action action);
4. get_report_action (uvm_severity severity, string message_ID);

Problem with uvm_reporting:

        UVM reporting also have some problem. While we used uvm reporting it print in a standard messaging format which has some redundant and unwanted parts and cause to very massive log generated as shown below.

UVM_INFO @ 0: reporter [RNTST] Running test test_1...
UVM_INFO testbench.sv(14) @ 0: uvm_test_top.pr [Message_id_1] message_1
UVM_INFO testbench.sv(18) @ 0: uvm_test_top.pr [Message_id_0] message_0
        

        In above message format having several information like the type of message, where to generate, line number, time, path, message ID, and message. 

How to modify standard message format:

        In beginning, we write one feature of UVM reporting is flexible. To achieve this and resolve above mention problem uvm give some class and inbuild method. UVM has uvm_report_sever which provides global support for reporting. If any change in uvm_report_server it will reflect in all test bench components. Therefore we extend this class and change as per over requirement.

        uvm_repot_sever having several methods. we can directly use this in our extended class and modified the standard message format. For that used compose_message() method.

virtual function string compose_message (uvm_severity severity, 

                                                                    string name, 

                                                                    string id, 

                                                                    string message, 

                                                                    string filename, 

                                                                    int line)

Example:

virtual function string compose_message( uvm_severity severity,string name,string id,string message,string filename,int line );

    $display("New Format: \n %0s",super.compose_message(severity,name,id,message,filename,line));

    $display("This is from uvm_report_sever extended class");

endfunction

        Apart from changing message format, we can do so many things using uvm_report_sever like customize a report summary, dump report status, summarize a report. For more methods click here.

uvm_report_catcher:

        uvm_report_catcher is a very useful tool while we work with error injection in a test bench. we look at one example. let's assume we write one error case and apply it to a test bench. An obvious Test bench has given and an error message during simulation but we intently put an error in the test case so it is right and print as a pass case. but it does not happen for that we required uvm_report_catcher. It is also used to change severity. Apart from there are so many scenarios to apply report_catcher. 

        uvm_report_catcher is used to catch messages from uvm_report_server. It is used or builds on callbacks object so all facilities in uvm_callback classes are available for registering catchers and controlling catcher state. uvm_report_catcher used as user extended class like uvm_report_server but in 

uvm_report_catcher must implement catch method in which the action to be taken on catching the report is specify. the catch method can return CAUGHT, in which case further processing of report is immediately stopped, or return THROW is which case the report is passed on other registered catchers. For used define extended uvm_report_catch always return action_e for catch() method. Apart from that inside catch() method, we can use the following method to customize the catcher.

get_severity: Returns the uvm_severity of the message that is currently being processed. 

get_verbosity: Returns the verbosity of the message that is currently being processed. 

get_id: Returns the string id of the message that is currently being processed. 

get_message: Returns the string message of the message that is currently being processed.

set_severity: Change the severity of the message to severity. 

set_verbosity: Change the verbosity of the message to verbosity. 

set_message: Change the text of the message to message. 

set_action: Change the action of the message to action. 

    Apart from above also used : get_context, get_client, get_action, get_fname, get_line, get_element_container, set_id, set_context, add_int, add_string and add_object.

            There are a number of reporting and debugging methods also provided by uvm like uvm_report_info, uvm_report_error, uvm_report_warning, uvm_report_fatal, uvm_report, summarize, etc. For more methods and functions click here.

Example: 

function action_e catch();

  //demote error to warning

  if((get_severity == UVM_FATAL) & (get_id == "DRIVER"))

    begin

      set_severity(UVM_ERROR);

      set_id("DRV_DEMOTED");

      set_message("Configuration FATAL demoted to ERROR::");

    end

 //To avoid all info and warnings messages

  if(get_severity == UVM_INFO & (get_severity == UVM_WARNING))

    begin

      set_action(UVM_NO_ACTION);

    end

  return THROW;

endfunction

Reporting in File:

            Create a log is very critical in any test bench. For that, uvm provides an action call UVM_LOG which is set through the set_*_action method. Before that, we need to open a file or create a new file. For that, we can use a system task calls $fopen(file_name, action_perform).

            We can compile action as well using simple | (or) of them like UVM_COUNT | UVM_EXIT. Note that UVM_COUNT, UVM_EXIT are used only for uvm_error and uvm_fatal. 

            UVM also provide flexibility to distinguish different file based on message severity, ID or both. For that used following methods. 

set_report_default_file (file_name);

set_report_severity_file(UVM_SEVERITY, file_name);

set_report_id_file(string id, file_name);

set_report_severity_id_file(UVM_SEVERITY, string id, file_name);

        At last, in the final stage of simulations, all files must be close using system task $fclose(file_name). 

Sample Code

class my_report_server extends uvm_report_server;

  `uvm_object_utils(my_report_server)

  //new function

  function new(string name="my_report_server");

    super.new(name);

    $display( "Customize report serevr %0s",name);

  endfunction : new

  //compose new message format. Return type is string  

   function string compose_message( uvm_severity severity,

                                           string name,

                                           string id,

                                           string message,

                                           string filename,

                                           int line );

     //old format

    //$display("old Format: \n %0s",super.compose_message(severity,name,id,message,filename,line));

     //Avoid filename and line number

     $display("NEW FORMAT:\t %0s %0s %0s %0s",severity, name, id, message);    

     //This display comes after every message.

     //$display("This is from uvm_report_sever extended class");

   endfunction

endclass: my_report_server

class demoter extends uvm_report_catcher;

  function new(string name = "demoter");

    super.new(name);

  endfunction  

  function action_e catch();

    //demote error to fatal

    //detect UVM_FATAL with Message_id_2 and change to UVM_ERROR message

    if((get_severity == UVM_FATAL) & (get_id == "Message_id_2"))

      begin

        set_severity(UVM_ERROR);

        set_id("Message_id_2 DEMOTER");

        set_message("Configuration FATAL demoted to ERROR::");

      end

   //To avoid all info and warnings messages

    if(get_severity == UVM_INFO & (get_severity == UVM_WARNING))

      begin

        set_action(UVM_NO_ACTION);

      end

    return THROW;

  endfunction

endclass: demoter

class print_reporting extends uvm_component;

  `uvm_component_utils(print_reporting)  

  function new(string name = "print_reporting", uvm_component parent);

    super.new(name, parent);

  endfunction: new

  function void print();

    `uvm_info("Message_id_1", "message_1 (uvm_low)", UVM_LOW)

    `uvm_info("Message_id_1", "message_2 (uvm_high)", UVM_HIGH)

    `uvm_info("Message_id_2", "message_3 (uvm_full)", UVM_FULL)

    `uvm_info("Message_id_2", "message_4 (uvm_debug)", UVM_DEBUG)

    `uvm_info("Message_id_0", "message_0 (uvm_none)", UVM_NONE)

    `uvm_warning("Message_id_1", "WARNING_message_1")

    `uvm_warning("Message_id_2", "WARNING_message_2")

    `uvm_error("Message_id_1", "ERROR_message_1")

    `uvm_error("Message_id_2", "ERROR_message_2")

    `uvm_fatal("Message_id_1", "FATAL_message_1")

    `uvm_fatal("Message_id_2", "FATAL_message_2")

  endfunction: print  

endclass: print_reporting

class test_1 extends uvm_test;

  `uvm_component_utils(test_1)

 //declearation  variable

  int base_log;  

  int error_log;

  int id_log;

  int error_id_log;

  //my_report_server rp_sv;

  demoter dmt;

  print_reporting pr;

  function new(string name="test_1", uvm_component parent=null);

    super.new(name, parent);

  endfunction: new

  //create required objects and add callback.

  function void build_phase(uvm_phase phase);

    super.build_phase(phase);

    //rp_sv = new("my_report_server");

    dmt = new();

    uvm_report_cb::add(null,dmt);

    //uvm_report_server::set_server(rp_sv);

    pr = print_reporting::type_id::create("pr", this);

  endfunction: build_phase

  function void start_of_simulation_phase(uvm_phase phase);

    super.start_of_simulation_phase(phase);

    //file open

    base_log = $fopen("base_log", "w");

    error_log = $fopen("error_log", "w");

    id_log = $fopen("id_log", "w");

    error_id_log = $fopen("error_id_log", "w");  

    //check file hendler assertion

    assert(base_log);

    assert(error_log);

    assert(id_log);

    assert(error_id_log);

    //set verbosity with different id and severity

    //To execute used cmd with +define+name

    `ifdef CMD_OFF_VLEVEL

      pr.set_report_verbosity_level(UVM_HIGH);

    `elsif CMD_OFF_IDVLEVEL

      pr.set_report_id_verbosity("Message_id_1", UVM_LOW);

    `elsif CMD_OFF_SIDVLEVEL

      pr.set_report_severity_id_verbosity(UVM_INFO, "Message_id_2", UVM_FULL);

    `endif

    //set actions

    pr.set_report_severity_action(UVM_INFO, UVM_DISPLAY | UVM_LOG);

    pr.set_report_severity_action(UVM_WARNING, UVM_DISPLAY | UVM_LOG);

    pr.set_report_severity_action(UVM_ERROR, UVM_DISPLAY | UVM_LOG);

    pr.set_report_severity_action(UVM_FATAL, UVM_DISPLAY | UVM_LOG);

    //set individuals action with file handlers

    pr.set_report_default_file(base_log);

    pr.set_report_severity_file(UVM_ERROR, error_log);

    pr.set_report_id_file("Message_id_1", id_log);

    pr.set_report_severity_id_file( UVM_ERROR, "Message_id_1", error_id_log );

  endfunction: start_of_simulation_phase

  //print pr object

  task run_phase(uvm_phase phase);

    super.run_phase(phase);

    pr.print();

  endtask:run_phase

  //close file using file handle

  function void final_phase( uvm_phase phase );

    $fclose(base_log);

    $fclose(error_log);

    $fclose(id_log);

    $fclose(error_id_log);

  endfunction: final_phase

endclass: test_1

module test_reporting;

  initial begin

    run_test("test_1");

  end

endmodule: test_reporting

Notes: 1) Apply all code at once also valid. If required to change or disable any component then comment on that part in a test class.

            2) For set verbosity either used direct command-line argument or used +define+CMD_OFF_VLEVEL or other options given in code.

OUTPUTS

Figure-1: Output of uvm_report_server. customize compose_message.

Figure-2: Output of before apply uvm_report_catcher

Figure-3: Output after applying uvm_report_catcher