Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User aggregators #95

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

User aggregators #95

wants to merge 31 commits into from

Conversation

nmtiwari
Copy link

@nmtiwari nmtiwari commented Feb 2, 2017

User Defined Aggregation support

@hridesh
Copy link
Member

hridesh commented Feb 2, 2017

Thanks for your work on this. Could you please review whether this pull request also contains changes that are part of the other pull request on local evaluation? I see several changes that are part of that previous pull request. If that is the case, then perhaps we can work on that first to get that merged and then come back to this pull request.

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This folder (.idea) should be ignored. Does this branch need to merge master?

README.md Outdated
@@ -0,0 +1,36 @@
#BoaEvaluator: Local Mode Boa Compiler
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this - it is in the other pull request

@@ -0,0 +1,172 @@
p: Project = input;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move this to 'test/known-good/' ? We dont have an examples directory and if you are going to put code examples in there, might as well use them as test cases.

@@ -102,7 +103,12 @@ public static void main(final String[] args) throws IOException {
jarName = className + ".jar";

// make the output directory
final File outputRoot = new File(new File(System.getProperty("java.io.tmpdir")), UUID.randomUUID().toString());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, these changes are part of the other pull request

private boolean possibleInBuiltAgg(String name) {
Set<String> names = aggregators.keySet();
for (final String entry: names) {
if(entry.contains(name)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be contains()? so if I have a built in named 'top' and I try to use an aggregator 'op' it matches? I think you just want a simple aggregators.containsKey(name) for this function.

@@ -280,7 +305,6 @@ public void visit(final TupleType n) {
st.add("name", tupType.toJavaType());
st.add("fields", fields);
st.add("types", types);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why removed?

@@ -33,7 +33,7 @@ public static void main(String[] args) {
// Arrays.sort(files, new Comparator<File>() {
// @Override
// public int compare(File f1, File f2) {
// int n1 = getNumber(f1.getName()), n2 = getNumber(f2.getName());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are commented code being changed?

@@ -16,26 +16,164 @@
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

part of the other pull request?

new Path(context.getConfiguration().get("boa.ast.dir", context.getConfiguration().get("boa.input.dir", "repcache/live")),
new Path("ast")));
if(DefaultProperties.localDataPath != null) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

part of other pull request?

@@ -38,6 +38,7 @@
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.util.Tool;

import boa.datagen.DefaultProperties;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

nmtiwari added 2 commits February 12, 2017 01:23
… has elements of only one type.

Adding matrix operations over tuples.

Adding a function to convert tuple into array if possible

Adding matrix operations into Symbol table

Fixing Test for Complex Array types
@nmtiwari nmtiwari force-pushed the user_Aggregators branch 2 times, most recently from 315acd7 to 8455d9d Compare February 13, 2017 07:18
…t operations

adding libs for supporting these operations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants