Skip to content

Commit

Permalink
Add Bazel rule for java_cup (#6)
Browse files Browse the repository at this point in the history
* Add `//cup:cup.bzl`
* Import `//third_party/cup` from java-cup-bin-11b-20160615.tar.gz at http://www2.cs.tum.edu/projects/cup/
* Add example `//java/jflex/examples/calculator` (and related javatests)
* Add `//third_party/com/google/truth` to make tests more elegant
* Update documentation
  • Loading branch information
regisd committed Oct 15, 2018
1 parent 1b59aa3 commit 7311e16
Show file tree
Hide file tree
Showing 21 changed files with 575 additions and 46 deletions.
70 changes: 32 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,23 @@
# Bazel rules for JFlex
# Bazel rules for JFlex & Cup

Rules to generate java source files from a lexer specification, by [JFlex][gh-jflex],
for projects using the [Bazel build system][bazel].
This repository offers two rules for projects using the [Bazel build system][bazel]:

Master: [![Build Status](https://api.cirrus-ci.com/github/jflex-de/bazel_rules.svg)](https://cirrus-ci.com/github/jflex-de/bazel_rules)
- Rule to generate java source files from a lexer specification, with [JFlex][gh-jflex]

- Rule to generate java source files from a parser specification, with [CUP][cup]

## Project health

Status of the **master** branch:
<a href="https://cirrus-ci.com/github/jflex-de/bazel_rules">
<img src="https://api.cirrus-ci.com/github/jflex-de/bazel_rules.svg" alt="Build status" height="20">
</a>

## Disclaimer

This is not an officially supported Google product.

## Preparation
### Update your workspace
## Prepare your Bazel workspace

Load the **bazel_rules** in your [`WORKSPACE` file][be_workspace]:

Expand All @@ -24,61 +31,48 @@ Load the **bazel_rules** in your [`WORKSPACE` file][be_workspace]:

load("@jflex_rules//jflex:deps.bzl", "jflex_deps")

# If you want to use JFlex.
jflex_deps()


## Usage
## Usage in BUILD files

load("@jflex_rules//jflex:jflex.bzl", "jflex")
load("@jflex_rules//cup:cup.bzl", "cup")

jflex(
name = "", # Choose a rule name
srcs = [], # Add input lex specifications
outputs = [], # List expected generated files
srcs = [], # A list of flex specifications
outputs = [], # List of expected generated files
)

cup(
name = "", # Choose a rule name
src = "", # Grammar specification
)

Then, this rule can be used as one of the `srcs` of another rules, such as a `java_library`.

### Attributes

* **name** (Name; required)
Unique name for this target.
* **srcs** (List of labels; required)
List of flex specifications.
* **outputs** (list of labels; required)
List of the generated java files.
* **skeleton** (Label; optional)
Skeleton use by the JFlex Emitter. **Only use this option if you know what you are doing.**

As usual, these rules can be used as one of the `srcs` of another rules, such as a `java_library`.

### Example
For more details, see [cup](cup) and [jflex](jflex).

The example **//java/jflex/examples/helloworld** generates a lexer from `helloworld.flex` with:

jflex(
name = "gen_hello_lexer",
srcs = ["helloworld.flex"],
outputs = ["HelloWorld.java"],
)

See [//java/jflex/examples/helloworld](/java/jflex/examples/helloworld) for more information.

## Directory layout
```
├── assets → assets for the web site
├── cup → contains the `cup.bzl` Skylark extension
├── java → main Java source code
│ └── jflex
│ └── examples → examples
├── javatests → tests
├── jflex → contains the `jflex.bzl`
│ ├── calculator → integration of JFlex and CUP
│ └── helloworld → simple lexer
├── javatests → tests of the examples
├── jflex → contains the `jflex.bzl` Skylark extension
└── third_party → Aliases for third-party libraries
└── com
└── google
└── guava
```


[bazel]: http://bazel.build/
[gh-jflex]: https://github.com/jflex-de/jflex
[cup]: http://www2.cs.tum.edu/projects/cup/
[be_maven_jar]: https://docs.bazel.build/versions/master/be/workspace.html#maven_jar
[be_workspace]: https://docs.bazel.build/versions/master/tutorial/java.html#set-up-the-workspace
3 changes: 3 additions & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,8 @@
# See https://docs.bazel.build/versions/master/build-ref.html#workspace

load("//jflex:deps.bzl", "jflex_deps")
load("//third_party:third_party_deps.bzl", "third_party_deps")

jflex_deps()

third_party_deps()
8 changes: 8 additions & 0 deletions cup/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Copyright 2018 Google LLC.
# SPDX-License-Identifier: Apache-2.0

package(default_visibility = ["//visibility:public"])

# This rule is license under Apache 2
licenses(["notice"]) # Apache 2

28 changes: 28 additions & 0 deletions cup/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Skylark rule for Java CUP


## Attributes

* **name** (Name; required)
Unique name for this target.
* **src** (Label; required)
The CUP specification.
* **parser** (String; optional)
The class name of the parser to generate. Defaults to `parser`, hence generating `parser.java`.
Note that the lower case is used to behave like the CUP program.
* **symbols** (String; optional)
The class name of the symbols holder. Defaults to `sym`, hence generating `sym.java`.
Note that the lower case is used to behave like the CUP program.

## Example

The example **//java/jflex/examples/helloworld** generates a lexer from `helloworld.flex` with:

cup(
name = "gen_parser",
src = "calculator.cup",
parser = "CalculatorParser",
symbols = "Calc",
)

See [//java/jflex/examples/helloworld](../java/jflex/examples/calculator).
30 changes: 30 additions & 0 deletions cup/cup.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""Bazel rules for cup. """

# CUP can only read from stdin, which Skylark rules don't support. Use a genrule for now.
def cup(name, src, parser = "parser", symbols = "sym", interface = False):
"""Generate a parser with CUP.
Args:
name: name of the rule.
src: the cup specifications.
parser: name of the generated parser class.
symbols: name of the generated symbols class.
interface: whether to generate an interface.
"""
opts = [
"-parser",
parser,
"-symbols",
symbols,
]
if interface:
opts = opts + ["-interface"]
options = " ".join(opts)
cmd = ("$(location //third_party/cup:cup_bin) -destdir $(@D) " + options + " < $<")
native.genrule(
name = name,
srcs = [src],
tools = ["//third_party/cup:cup_bin"],
outs = [parser + ".java", symbols + ".java"],
cmd = cmd,
)
29 changes: 29 additions & 0 deletions java/jflex/examples/calculator/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Copyright 2018 Google LLC.
# SPDX-License-Identifier: Apache-2.0

package(default_visibility = ["//visibility:public"])

load("//jflex:jflex.bzl", "jflex")
load("//cup:cup.bzl", "cup")

java_library(
name = "calculator",
srcs = ["CalculatorParserException.java",":gen_parser", ":gen_lexer"],
deps =[
"//third_party/cup", # the runtime would be sufficient
]
)

cup(
name = "gen_parser",
src = "calculator.cup",
parser = "CalculatorParser",
symbols = "Calc",
)

jflex(
name = "gen_lexer",
srcs = ["calculator.flex"],
outputs = ["CalculatorLexer.java"],

)
11 changes: 11 additions & 0 deletions java/jflex/examples/calculator/CalculatorParserException.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
// Copyright 2018 Google LLC.
// SPDX-License-Identifier: Apache-2.0

package jflex.examples.calculator;

/** An exception from the lexer/parser. */
public class CalculatorParserException extends RuntimeException {
CalculatorParserException(String message) {
super(message);
}
}
75 changes: 75 additions & 0 deletions java/jflex/examples/calculator/calculator.cup
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
package jflex.examples.calculator;

import java_cup.runtime.Symbol;
import java.util.Vector;
/*
* Copyright (C) 2018 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/


/**
* Parser for a simple calculator.
*
* <p>The grammar defines a simple arihmetic expressions that supports the addition,
* the multiplication, and parentheses.
*
* @author Régis Décamps
*/

parser code {:
:}

// Terminals: () * +
terminal LPAR, RPAR;
terminal MULTIPLY;
terminal PLUS;
// A number is also a terminal, bit it's of type Integer.
terminal Integer NUMBER;

non terminal Integer expr;
non terminal Integer multiply_expr;
non terminal Integer term;

precedence left PLUS;
precedence left MULTIPLY;

// This is where the grammar starts.

// An expression is a sum of priority_expr. Because the parser needs to produce priority_expr in
// in order to assemble expr, this effectively makes priority_expr of higher priority than
// than expr in the computation of the result.
// We also implement the actual computation done by the calculator. RESULT is the label that is
// assigned automatically to the rhs, in this case 'expr' because 'expr ::= ...'.
// Since the priority_expr is an object of type Integer, we can call intValue() to retrieve its
// value.
expr ::=
expr:a PLUS multiply_expr:b
{: RESULT = a.intValue() + b.intValue(); :}
| multiply_expr:e
{: RESULT = e; :}
;
multiply_expr ::=
multiply_expr:a MULTIPLY term:b
{: RESULT = a.intValue() * b.intValue(); :}
| term:t
{: RESULT = t; :}
;
// The last term closes the grammar. It's a number, or another expression in parentheses.
// Note that local identifiers don't need to be a single letter. We use "nb" for the terminal
// number.
term ::=
LPAR expr:e RPAR {: RESULT = e; :}
| NUMBER:nb {: RESULT = nb; :}
;
88 changes: 88 additions & 0 deletions java/jflex/examples/calculator/calculator.flex
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
/*
* Copyright (C) 2018 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package jflex.examples.calculator;

import java_cup.runtime.Symbol;

/**
* A simple lexer/parser for basic arithmetic expressions.
*
* @author Régis Décamps
*/

%%


%public
%class CalculatorLexer
// Use CUP compatibility mode to interface with a CUP parser.
%cup

%unicode

%{
/** Creates a new {@link Symbol} of the given type. */
private Symbol symbol(int type) {
return new Symbol(type, yyline, yycolumn);
}

/** Creates a new {@link Symbol} of the given type and value. */
private Symbol symbol(int type, Object value) {
return new Symbol(type, yyline, yycolumn, value);
}
%}

// A (integer) number is a sequence of digits.
Number = [0-9]+

// A line terminator is a \r (carriage return), \n (line feed), or \r\n. */
LineTerminator = \r|\n|\r\n

/* White space is a line terminator, space, tab, or line feed. */
WhiteSpace = {LineTerminator} | [ \t\f]


%%

// This section contains regular expressions and actions, i.e. Java code, that will be executed when
// the scanner matches the associated regular expression.


// YYINITIAL is the initial state at which the lexer begins scanning.
<YYINITIAL> {

/* Create a new parser symbol for the lexem. */
"+" { return symbol(Calc.PLUS); }
"*" { return symbol(Calc.MULTIPLY); }
"(" { return symbol(Calc.LPAR); }
")" { return symbol(Calc.RPAR); }

// If an integer is found, return the token NUMBER that represents an integer and the value of
// the integer that is held in the string yytext
{Number} { return symbol(Calc.NUMBER, Integer.parseInt(yytext())); }

/* Don't do anything if whitespace is found */
{WhiteSpace} { /* do nothing with space */ }
}

// We have changed the default symbol in the bazel `cup()` rule from "sym" to "Calc", so we need to
// change how JFlex handles the end of file.
// See http://jflex.de/manual.html#custom-symbol-interface
<<EOF>> { return symbol(Calc.EOF); }

/* Catch-all the rest, i.e. unknow character. */
[^] { throw new CalculatorParserException("Illegal character <" + yytext() + ">"); }
Loading

0 comments on commit 7311e16

Please sign in to comment.