cross-compile the benchmark in compiler Gym on X86 platform to RISCV platform

[sparks-code]

❓ Questions and Help

Hello, if I want to cross-compile the benchmark in compiler Gym on X86 platform to RISCV platform, what do I need to modify?

Additional Context

[ChrisCummins]

Hey @sparks-code sorry for the slow reply!

Benchmark's have a command line invocation that is used to compile the bitcode down to a binary. If all you need to do is change some flags in the clang invocation, then you can do that by:

On the other hand, if you need to recompile the C/C++ sources to bitcode, that may be a bit more tricky. Only a couple of the benchmark datasets ship as source files, the rest ship as bitcodes.

Can I ask what your use case is, and if there are any benchmarks in particular that you are interested in? That may help me give better suggestions.

Cheers,
Chris

[sparks-code]

@ChrisCummins Thank you very much for your reply.

I want to use compilerGym for code optimization on the RISCV platform,so I need to recompile the C/C++ source code to bitcode.

Currently I am trying to recompile cbench-v1.1, can I ask if you you have any recommended relatively easy benchmarks to recompile? Or what else I need to do？

I am waiting for your suggestions.

[ChrisCummins]

You could take a look at Anghabench. It's 1M C functions that are shipped as source code and compiled to IR on-demand. You could hack on the command line here to compile to RISCV:

CompilerGym/compiler_gym/envs/llvm/datasets/anghabench.py

Lines 105 to 112 in 8d25c10

	compile_cmd = ClangInvocation.from_c_file(
	c_file_abspath,
	copt=[
	"-ferror-limit=1", # Stop on first error.
	"-w", # No warnings.
	],
	).command(outpath=tmp_path)
	subprocess.check_call(compile_cmd, timeout=300)

CSmith works largely the same:

CompilerGym/compiler_gym/envs/llvm/datasets/csmith.py

Lines 213 to 227 in 8d25c10

	# Compile to IR.
	with Popen(
	self.clang_compile_command,
	stdin=subprocess.PIPE,
	stdout=subprocess.PIPE,
	stderr=subprocess.DEVNULL,
	) as clang:
	stdout, _ = communicate(clang, input=src, timeout=300)
	if clang.returncode:
	compile_cmd = " ".join(self.clang_compile_command)
	raise BenchmarkInitError(
	f"Compilation job failed!\n"
	f"Csmith seed: {seed}\n"
	f"Command: {compile_cmd}\n"
	)

For CBench, I use this script to compile each of the benchmarks's sources to IR. You could adapt that:

CompilerGym/compiler_gym/third_party/cbench/make_llvm_module.py

Lines 20 to 35 in 8d25c10

	def make_cbench_llvm_module(
	benchmark_dir: Path, cflags: List[str], output_path: Path
	) -> str:
	"""Compile a cBench benchmark into an unoptimized LLVM bitcode file."""
	src_dir = benchmark_dir / "src"
	if not src_dir.is_dir():
	src_dir = benchmark_dir
	assert src_dir.is_dir(), f"Source directory not found: {src_dir}"
	src_files = [path for path in src_dir.iterdir() if path.name.endswith(".c")]
	assert src_files, f"No source files in {src_dir}"
	benchmark = make_benchmark(inputs=src_files, copt=cflags or [])
	# Write just the bitcode to file.
	with open(output_path, "wb") as f:
	f.write(benchmark.proto.program.contents)

Hope that helps!

Cheers,
Chris

[sparks-code]

import gym
import compiler_gym

env = gym.make("llvm-autophase-ic-v0")
dataset = env.datasets.dataset("benchmark://cbench-v1")

benchmark=env.datasets.benchmark("benchmark://cbench-v1/bzip2")
(benchmark.proto.dynamic_config.build_cmd.argument)
def eval_action(env, action: int):
    with env.fork() as fkd:
        return (fkd.step(action)[1], action)

for action in range(env.action_space.n):
    print(eval_action(env,action))

hello, @ChrisCummins ,thank you for your help.I have compiled cbench to riscv platform ，upgraded llvm to 17 and made a corresponding replacement of the action space，I use the python script above to test my work is correct. But the output looks like this

(0.0, 0)
(0.006172839506172839, 1)
(0.0, 2)
(0.0, 3)
(0.0, 4)
(0.12962962962962962, 5)
(0.10493827160493827, 6)
(0.0, 7)
(0.006172839506172839, 8)
(-0.09259259259259259, 9)
(0.11728395061728394, 10)
(0.0, 11)
(0.0, 12)
(-0.006172839506172839, 13)
(0.0, 14)
(0.0, 15)
(0.0, 16)
(0.0, 17)
(0.0, 18)
(0.0, 19)
(0.006172839506172839, 20)
(0.04938271604938271, 21)
(0.0, 22)
(0.9567901234567902, 23)
(0.2654320987654321, 24)
...
(0.0, 33)
(0.0, 34)
(0.0, 35)
(0.0, 36)

---

ServiceIsClosed Traceback (most recent call last)
/root/CompilerGym/examples/llvm_autotuning/test_riscv.ipynb 单元格 4 in ()
11 return (fkd.step(action)[1], action)
13 for action in range(env.action_space.n):
---> 14 print(eval_action(env,action))

/root/CompilerGym/examples/llvm_autotuning/test_riscv.ipynb 单元格 4 in eval_action(env, action)
9 def eval_action(env, action: int):
---> 10 with env.fork() as fkd:
11 return (fkd.step(action)[1], action)

File ~/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/envs/llvm/llvm_env.py:655, in LlvmEnv.fork(self)
654 def fork(self):
--> 655 fkd = super().fork()
656 if self.runtime_observation_count is not None:
657 fkd.runtime_observation_count = self.runtime_observation_count

File ~/anaconda3/envs/compiler_gym/lib/python3.8/site-packages/compiler_gym/service/client_service_compiler_env.py:525, in ClientServiceCompilerEnv.fork(self)
523 request = ForkSessionRequest(session_id=self._session_id)
524 try:
--> 525 reply: ForkSessionReply = self.service(
526 self.service.stub.ForkSession, request
527 )
529 # Create a new environment that shares the connection.
...
209 # We raise "from None" to discard the gRPC stack trace, with the
210 # remaining stack trace correctly pointing to the CompilerGym
211 # calling code.

ServiceIsClosed: RPC communication failed because channel is closed

It seems that when some action apply to the benchmark the environment crashed，so my question is, what should i do to ensure my action space is correct ?

[ChrisCummins]

It seems that when some action apply to the benchmark the environment crashed，so my question is, what should i do to ensure my action space is correct ?

I would start by narrowing down the cause of the crash: Is it always the same action? Is it always on the same benchmark? Once you have figured out which action(s) are at fault it may be worth digging into the LLVM docs to ensure that the pass is being used as expected.

Cheers,
Chris