Overview:
Running automated regression on a daily basis as part of the daily build is inevitable nowadays! But it is very painful to maintain hundreds of automated tests & remote parallel test execution results!
In this article, lets see how we can use TestArchiver + PostgreSQL + Grafana to archive all our robot framework test results and create an advanced robot framework test execution dashboard.
In future article we will use the archived test results for our AI driven analysis/predictions.
I already have been using MySQL + Grafana for storing my robot framework test execution results. You can find more details in RF + MySQL + Grafana
Key Benefits:
- Unified storage for the test run results
- Archiving your test results to PostgreSQL database
- Meeting the test data challenge for AI and ML
Metrics Covered:
- Execution Status – Latest
- Total Executions – Over Time Period
- Average Pass and Failure Percentage
- Top 5 Failed Test Cases
- All Execution Status Over Time Period
- Average Execution Time
- Execution Trend Graph
Grafana & PostgreSQL Integration:
PostgreSQL Setup:
- Download PostgreSQL – Version > 9.5
- Use the following link to install PostgreSQL DB
- Make sure PostgreSQL server is up and running
- Connect to PostgreSQL database server using PgAdmin
- Create a new database schema robot_results using PgAdmin
Grafana Setup:
- Download Grafana – Version > 7.0
- Use the following link to setup Grafana
- Start the Grafana server
- Launch browser with this URL http://localhost:3000 to access the Grafana home page

- Update Grafana data source to point to the PostgreSQL instance
- Click on Configuration Menu in the Left Panel and Select Datasource
- Click on Add data source and select PostgreSQL database
- Enter PostgreSQL server details as below and click on Save & Test. The connection should be successful.

Once integrating PostgreSQL and Grafana is completed, then the next step is to send the robot test results data to PostgreSQL DB and enable the default Grafana dashboard.
Robot Framework & PostgreSQL Integration:
Well at this point, I assume that you have Python 3+, Robot Framework installed already and you are running your automated test cases.
Additionally, if you have few robot test results output.xml files then it would be very helpful for you to follow the below details.
Python Library Setup:
We will be using TestArchiver library in order to parse the robot framework output.xml to PostgreSQL database. Download the entire project using the following link TestArchiver-Git
TestArchiver:
TestArchiver is a tool for archiving your test results to a SQL/PostgreSQL database. This way the future `Robot Framework` related tools and plugins will have a unified storage for the test run results.
I would like to say Thank you for SALabs team for the effort to build such a useful library to support different testing frameworks. Please refer the below link if you would like to learn more.. TestArchiver-Git
Create DB Config File:
Sample configuration details for your reference. Update the values as per your PostgreSQL setup. Please save the file with name test_config.json under TestArchiver-master folder.
Adding Application Name as Metadata:
As each execution is for an application or project, we will add the application/project details with –metadata option when we trigger the robot test execution. Please the below command or include the application in using -M with the existing robot command line option.
Push Test Results to PostgreSQL DB:
Now we are all set to push the report to PostgreSQL DB. Please use the below command to parse the details in output.xml and push it to DB.
Additionally, we have used 2 more parameters to distinguish the test execution while parsing the output.xml. In our pre-defined dashboard we have used both the values as drop down options, in case if you would like to ignore these parameters then the dashboard queries needs to be customized accordingly.
–team TEAM Team name for the test series (Eg: QA_Team)
–series SERIES Name of test series (and Optionally build number – ‘SERIES_NAME#BUILD_NUM’)
Example:
Below is the sample command line code after replacing all the parameters specific to project for reference.
Create View in PostgreSQL DB:
Alright, at this point you would have successfully pushed the robot framework results to PostgreSQL DB. When you check the database you will see the below tables created automatically and parse results will be stored in respective tables.

Finally, we will create a view in PostgreSQL DB. This is very important step as the default Grafana dashboard rely only on this view. Check out the following link as reference to create views in db – How to create views in PostgreSQL.
View Name: automation_execution_results
Use following query to create the new view automation_execution_results:
Robot Framework Test Results in Grafana Dashboard:
Grafana Plugins:
We will have to install following grafana plugins to incorporate the robot framework dashboard.
- Click on Configuration Menu in the Left Panel and Select Plugins
- Click on Find more plugins Grafana.com and Search for below plugins
Select Panel under Plugin Type and Click on Pie Chart | D3 Guage
Please follow the instruction in the below plugin installation page and install the plugins


I have created a decent looking dashboard and published them under grafana dashboards. You can download the pre-build dashboards using the following link Robot Framework Grafana Dashboard
The downloaded zip file should also contain the Grafana dashboards (in JSON format). Import those dashboard files in Grafana (You can find the Import feature under Home as shown below).
Once it successfully imports, It will create dashboards as shown below.
Hope you like this article!! Please provide your rating and helps us to improve 🙂

QA Manager | Automation Solution Architect | RPA – Software Robot Developer |
“I have learned how to learn quicker”
I am passionate about learning modern technologies and implementing them in my existing workstream. I love to automate every task we repeatedly perform in the workplace… Read more on CognitiveQE
Very Good / User Friendly steps on Integrity of the DB Scripts with Robot Framework Keywords. I was able integrate easily using these steps.
Estou precisando de uma ajuda
python3 /Users/platoon/TestArchiver-master/test_archiver/output_parser.py –config /Users/platoon/ TestArchiver-master/db_config.json –format robotframework /Users/platoon/QA-Tests/Robotframework_CI/Tests/Canary/PersonAccounts/output.xml
usage: output_parser.py [-h] [–config CONFIG_FILE] [–dbengine DBENGINE]
[–database DATABASE] [–host HOST] [–user USER]
[–pw PW] [–port PORT] [–dont-require-ssl]
[–format {robot,robotframework,xunit,junit,mocha-junit,pytest-junit,mstest}]
[–repository REPOSITORY] [–team TEAM]
[–series SERIES] [–metadata METADATA]
[–change_engine_url CHANGE_ENGINE_URL]
output_files [output_files …]
output_parser.py: error: unrecognized arguments: /Users/platoon/QA-Tests/Robotframework_CI/Tests/Canary/PersonAccounts/output.xml
platoon@Platoon-Mac-mini ~ %
Trying to figure out why it doesn’t recognize the arguments. Also a small typo on your steps where you talk about creating the db file. You say to save as “test_config.json” but in your parameters it has “db_config.json”. It’s not a big deal as most users following these steps would figure that out.
Thanks Raul .. I will update test_config to db_config .. which parameter it’s not picking up please let me know
platoon@Platoon-Mac-mini ~ % python3 /Users/platoon/TestArchiver-master/test_archiver/output_parser.py –config /Users/platoon/TestArchiver-master/db_config.json –format robotframework /Users/platoon/QA-Tests/Robotframework_CI/Tests/Canary/PersonAccounts/output.xml
Traceback (most recent call last):
File “/Users/platoon/TestArchiver-master/test_archiver/output_parser.py”, line 735, in
connection = database_connection(config)
File “/Users/platoon/TestArchiver-master/test_archiver/archiver.py”, line 404, in database_connection
config[‘require_ssl’] if ‘require_ssl’ in config else True)
File “/Users/platoon/TestArchiver-master/test_archiver/database.py”, line 24, in __init__
self._connect()
File “/Users/platoon/TestArchiver-master/test_archiver/database.py”, line 93, in _connect
sslmode=’require’ if self.require_ssl else ‘prefer’,
File “/Users/platoon/Library/Python/3.7/lib/python/site-packages/psycopg2/__init__.py”, line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: server does not support SSL, but SSL was required
Some progress I guess, but now I’m seeing this error when trying to push the results into the database…
Here’s my database config –> https://a.cl.ly/Z4ukLAbr
Got it – I even had this issue .. in case if you are not using ssl .. then disable the ssl in the following line .. everything should work fine .. File “/Users/platoon/TestArchiver-master/test_archiver/database.py”, line 9
Thank you for sharing good content.
The same problem is occurring during setup.
If it’s database.py line9, pycopg2 = None, how can I change this?
I’d appreciate it if you could let me know.
Please share the error msg .. I can.. is it same ssl issue?
Comment that line and add a line sslmode = ‘disable’ it should work
I think I’m getting closer… All my data is in the database now I’m seeing this error in grafana… https://a.cl.ly/v1u2ejKR
And yes I created the views. But I’m having some trouble getting D3 gauge to work properly.
This error is due to data source connectivity please check and data source.. You have option to test the connection..
That worked!!!! Thanks https://a.cl.ly/rRu7l1Ab
That was only 1 run as I need to get more runs into the database to establish more data. Thanks again Sridhar for replying quickly to my every issue I had 👍.
Hi Raul .. In case if you have any interesting topic which you would like to publish feel free to reach out to me .. I can add you as author and the post will be under your name
Raul Please do review the article that will help us improve
File “C:\TestArchiver-master\test_archiver\output_parser.py”, line 735, in
connection = database_connection(config)
File “C:\TestArchiver-master\test_archiver\archiver.py”, line 399, in database_connection
return PostgresqlDatabase(config[‘database’],
File “C:\TestArchiver-master\test_archiver\database.py”, line 24, in __init__
self._connect()
File “C:\TestArchiver-master\test_archiver\database.py”, line 88, in _connect
self._connection = psycopg2.connect(
File “C:\Users\lotte\AppData\Local\Programs\Python\Python38-32\lib\site-packages\psycopg2\__init__.py”, line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: server does not support SSL, but SSL was required
my db_config.json
{
“db_engine”: “postgresql”,
“database”: “robot_results”,
“host”: “localhost”,
“port”: “5432”,
“user”: “postgres”,
“password”: “skskal”,
“require_ssl”: “False”
}
Thank you very much.
It’s been applied because you changed the part you told me.
Data has been registered in DB normally.
Thank you.
The only issue I’m having now is with my data not appending correctly on the dashboard. If you see my dashboard you can see that not all of my data is shown even though I’ve ran 1 suite 5-6 times…
—–> https://a.cl.ly/7KuLLjyJ
Here’s my database table —> https://a.cl.ly/6queeN2w
Vanilla dashboard is configured using the meta data (Application, Suite and Build No) .. I think issue is with meta data .. all 3 drop downs are none in your case ..
In case if you want to really skip there are options .. Please let me the background why you want to skip then I can share my ideas..
I figured it was the meta data, I want to demo the dashboard to our group and show the capabilities of Test Archiver and Grafana. I didn’t want to configure the dropdown just yet since I would need to also configure our Jenkins server. I want showcase our test cases. If there are options I can quickly implement great if not I’ll try and use the sample data.
Good after fixing your metadata if you still face issues let me know
Hi Raul once you are fully done .. If you like the post do click the like button on top and also provide your rating by clicking on stars ..
Hi Sridhar,
I am facing the following error. Kindly assist.
cmd line path>py Path\output_parser.py –config Path\test_config.json –dont-require-ssl –team Your_Team_Name –series jenkins_build_no#Jenkins-Your_Application_Build_No –format robotframework Path\output.xml
Traceback (most recent call last):
File “Path\TestArchiver\testarchiver\output_parser.py”, line 709, in
config = read_config_file(args.config_file)
File “Path\TestArchiver\testarchiver\archiver.py”, line 44, in read_config_file
return json.load(config_file)
File “Path\AppData\Local\Programs\Python\Python38-32\lib\json\__init__.py”, line 293, in load
return loads(fp.read(),
File “Path\AppData\Local\Programs\Python\Python38-32\lib\json\__init__.py”, line 357, in loads
return _default_decoder.decode(s)
File “Path\AppData\Local\Programs\Python\Python38-32\lib\json\decoder.py”, line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File “Path\AppData\Local\Programs\Python\Python38-32\lib\json\decoder.py”, line 355, in raw_decode
raise JSONDecodeError(“Expecting value”, s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 8 column 12 (char 145)
Hi problem is with the dB config file .. please use a online Json validator to cross check if it’s in right format
The above issue has been resolved and received an another issue.
The JSON file is:
{
“db_engine”: “postgresql”,
“database”: “foo”,
“host”: “localhost”,
“port”: “foo”,
“user”: “foo”,
“password”: foo”,
“sslmode”: “disable”
}
cmd line path>py Path\output_parser.py –config Path\test_config.json –dont-require-ssl –team Your_Team_Name –series jenkins_build_no#Jenkins-Your_Application_Build_No –format robotframework Path\output.xml
psycopg2.OperationalError: server does not support SSL, but SSL was required
In file \TestArchiver\testarchiver\ database.py search for sslmode .. comment that line and add a new line sslmode=‘disable’ ..
Hi Sridhar,
All steps worked fine . importing the dashboard only not working
thanks
Poovaraghan
What error message you are getting…
Hello
Posting the results into mysql database wouldn’t work as expected – we discuss about it yesterday – but it works fine with a postgres sql database.
Again, many thanks for your great work !
The only bug I found is the sort option above the graphics (Application Name , Suite Name, No Build).
When I select a specific Application Name, the sort won’t work as expected and I stand with all the information.
Hayden
Thank you Hayden ..
Hi Sridhar,
Could you also create the robot dashboard for the inflexdb?
HI Sridhar, could you please past the query to create VIEW for automation_execution_results
Hi Kishore – I did pasted the entire query
Hi Sridhar, not able to see query again. Could you please past it again
Hi Kishore.. let me know still if you face the issue ..
Hi Sridhar,
All my widgets are saying “No data to show” – Do I need a minimum number of test runs?
I’ve included the -M Application: as an argument in the test execution.
This is my testarchiver cmd:
testarchiver –config db_config.json –dont-require-ssl –team QA –series manually_run#1 output.xml
The drop-downs for Application Name, Suite Name and Build No are all populated in my grafana dashboard.
Also I noticed this warning message on the variables screen for the $build_no:
“This variable is not referenced by any variable or dashboard”
Grafana v7.4.3
Instead of manually_run use jenkins_build_no#1 .. jenkins_build_no is referenced in the dashboard variable section
Hi Sridhar,
Is it also possible to push the robot test results ( log/report.html) from the Azure pipeline to a Grafana Dashboard as in the example and description as described above?
Hi Easiest way is to parse the output.xml to db and display results in grafana.. Grafana wouldn’t accept html as data source .. Please provide few additional info I will try and will let you know
Hi Sirdhar,
I am trying to work with this tutorial and I am getting this error for Test_archiver – from . import archiver, configs
ImportError: attempted relative import with no known parent package. Any ideas what can I do ?
Hi Sridhar,
I am also facing issues while executing output_parser.py. Below is my command
python3 /usr/local/bin/TestArchiver/test_archiver/output_parser.py –config /usr/local/bin/TestArchiver/db_config.json –dont-require-ssl –team qa_team –format robotframework /home/rffiles/output.xml
I get import error for archiver even though I installed archiver package. Any pointers would be helpful.
File “/usr/local/bin/TestArchiver/test_archiver/output_parser.py”, line 8, in
from . import archiver, configs
ImportError: cannot import name ‘archiver’
Hai Sirdhar,
I am trying to work with this tutorial and I am getting this error for Test_archiver – from . import archiver, configs
OS MAC
python /Users/andikaprananto/TestArchive/test_archiver/output_parser.py –config /Users/andikaprananto/TestArchive/db_conifg.json –dont-require-ssl –team QA_Team –series Jenkins-Build-100 –format robotframework /Users/andikaprananto/PycharmProjects/AladinOtomasi-Master/results/output.xml
Traceback (most recent call last):
File “/Users/andikaprananto/TestArchive/test_archiver/output_parser.py”, line 8, in
from . import archiver, configs
ValueError: Attempted relative import in non-package
Please Help Me
Use Python3 not work
You can install testarchiver directly using pip install testarchiver.. I guess test archiver is not installed properly..
Hi Sridhar,
I’ve installed pip install testarchiver, but the error is still the same and can’t run
please check screenshoot me https://www.imagebam.com/view/ME80C51
Please Help Me…..
please check this is screenshot https://www.imagebam.com/view/ME80C5S
Hi Andika ,
What is the the solution for the above mentioned problem.
Please replace
” from . import xxx”
To
import xxx
This should solve the entire equation.
Hi Sridhar,
I’ve installed pip install testarchiver, but the error is still the same and can’t run
please check screenshoot me https://www.imagebam.com/view/ME80C51
Please Help Me…..
Hi Sridhar
Previously I had an issue with TestArchiver parsing data to PostgreSQL but it was solved. Now I’m having problem when in dashboard on Application Name | Suite Name | Build No only shows NONE please check screenshot: https://www.imagebam.com/view/ME826KJ
connection from PostgreSQL to Grafana OK, please check screenshot https://www.imagebam.com/view/ME826N4
Parse data from output.xml to PostgreSQL OK, please check screenshot https://www.imagebam.com/view/ME826N5
but why, dashboard is all empty dont data, please help me how to solve it please
Hi Sridhar
I have data only in ‘ Top 5 Failed Tests’ But everywhere else it’s empty.
Hi Jason Please provide the version of grafana and RF . It helps to check
Hi Sridhar ,
RF version is 5.0.1 and grafana version is 9.1.1
Hello Sridhar,
thank you for the detailed steps, I could successfully insert the data , but my dashboard not seems to be working properly.
it shows the dropdowns.
here is the screenshot for the reference, Please help me on this.
https://www.imagebam.com/view/MEFO9LQ
https://www.imagebam.com/view/MEFO9LR
Thank you Arjun .. Please perform some execution and check if data is not populated
thank your response.
I did insert data multiple times., but the dashboard does not show anything.
i see some variable error, please have a look
https://www.imagebam.com/view/MEFOAZZ
I am also planning to post with lastest Grafana and robot framework versio. Soon
Hi Sridhar,
Can please provide an estimate of when the new version will be uploaded?
Thank you Sridhar for the very informative article. Could you please share the screenshot of the PostgreSQL database table from where you are populating the data?
Actually, I am using Selenium web driver but I can insert the results in DB, and then I can use your dashboard for graphical representation.
hi..Can I configure gitlab pipline number rather than jenkins?
Hello Saridhar VP
I have successfully pushed the output.xml to ptsgresql with testarchiver, but when I use the dashboard template from Saridhar in grafana it gives this error, how can I fix it, thank you
https://prnt.sc/xgTFAeYsu09F
Hello Saridhar VP
Hello, I would like to ask, I have successfully pushed the output.xml data to PostgreSQL and into the table, but why when checked in the dashboard Application Name, Suite Name, Build No contains NONE, and the data cannot be loaded in the dashboard, can you help me?
Screenshot Dashboard : https://prnt.sc/R25YID9XXMyY
But, Table Suite_Metadata Null Data : https://prnt.sc/e2MD9o55a5QX