Bug 895

Summary:	openpower-isa should not require soc to run tests
Product:	Libre-SOC's first SoC	Reporter:	Jacob Lifshay <programmerjake>
Component:	Source Code	Assignee:	Luke Kenneth Casson Leighton <lkcl>
Status:	CONFIRMED ---
Severity:	normal	CC:	libre-soc-bugs, programmerjake
Priority:	---
Version:	unspecified
Hardware:	Other
OS:	Linux
NLnet milestone:	---	total budget (EUR) for completion of task and all subtasks:	0
budget (EUR) for this task, excluding subtasks' budget:	0	parent task for budget allocation:
child tasks for budget allocation:		The table of payments (in EUR) for this task; TOML format:

Description Jacob Lifshay 2022-07-27 20:10:43 BST

most of the test_caller_*.py files have:
from soc.simple.test.test_runner import TestRunner

imho if openpower-isa needs TestRunner, then TestRunner should be in openpower-isa or one of its dependencies. same for all the other soc imports.

.../openpower-isa$ rg 'from soc\.'
src/openpower/test/test_state_class.py
15:from soc.simple.test.teststate import HDLState

src/openpower/test/runner.py
33:from soc.config.test.test_loadstore import TestMemPspec

src/openpower/decoder/isa/test_caller_logical.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_fmvis.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_alu.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_bitmanip_av.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_svp64_alu.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_mul.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_shift_rot.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_svp64_logical.py
13:from soc.simple.test.test_runner import TestRunner

src/openpower/decoder/isa/test_caller_bitmanip.py
13:from soc.simple.test.test_runner import TestRunner

Comment 1 Luke Kenneth Casson Leighton 2022-07-28 05:06:46 BST

yep i know, that was a mistake by kyle which i haven't had time to correct.
it requires registration of the HDLState with the test API using a
Factory Class (aka "a python dictionary") by a name (string),
followed by that class being obtained *by name* (string) if it is
needed.

strictly speaking though, looking at them now, not one single one
of those test_caller_*.py files should have been committed to
the openpower-isa repository.

ah wait - see the "run_hdl=False" argument? they don't *actually*
run TestIssuer.

needs cleanup basically.

on the very long TODO list.

Comment 2 Jacob Lifshay 2022-07-28 12:29:36 BST

(In reply to Luke Kenneth Casson Leighton from comment #1)
> strictly speaking though, looking at them now, not one single one
> of those test_caller_*.py files should have been committed to
> the openpower-isa repository.

imho we need some kind of tests in openpower-isa.git for all the instructions in openpower-isa.git, so either we should keep them and make them independent of soc, or create new tests that run the simulator and check outputs.

Comment 3 Luke Kenneth Casson Leighton 2022-07-28 13:36:13 BST

(In reply to Jacob Lifshay from comment #2)

> imho we need some kind of tests in openpower-isa.git for all the
> instructions in openpower-isa.git, so either we should keep them and make
> them independent of soc, or create new tests that run the simulator and
> check outputs.

anyone following the discussion on IRC last year will know that that's
exactly what the test API is and always has been planned to be. also
from the discussions we have weekly, you will know that there is a plan
to incorporate the OPF Compliance Test Suites. this was discussed appx
6 months ago. it is held up simply because machine-readable versions of
the Compliance Suite are not available yet and i am not wasting time doing
PDF parsing. once was enough.

you'll also have seen yesterday and read the
proposal https://libre-soc.org/nlnet_2022_opf_isa_wg/ which includes
a budget for tidy-up and continuation of the Test API.

i also mention the Test API in one of the OpenPOWER Conferences, last year.

it was just inexperience that led kyle to add an incorrect dependency / insufficient abstraction and i haven't had time to fix it.