Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add INT8 type #1994

Merged
merged 1 commit into from
Sep 12, 2023
Merged

Add INT8 type #1994

merged 1 commit into from
Sep 12, 2023

Conversation

Ashleyhx
Copy link
Contributor

@Ashleyhx Ashleyhx commented Sep 6, 2023

Completing the numerical typing system - add INT8 dataType, so that entries of type INT8 can be stored into the table.
An example can be found in StudyAt.csv.

Copy link
Collaborator

@acquamarin acquamarin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good progress!
You may need to change the ValueVector,factorizedTable classes(which are used in query processing), so we can store the INT8 value in these internal data structures.
To add a test in kuzu:
We have a small dataset that used for testing: tinysnb (dataset/tinysnb). You can add a INT8 column in the vMovies.csv file. We usually write simple projection tests in this file: test/test_files/tinysnb/projection/single_label.test

Testing/Temporary/CTestCostData.txt Outdated Show resolved Hide resolved
Testing/Temporary/LastTest.log Outdated Show resolved Hide resolved
src/c_api/value.cpp Outdated Show resolved Hide resolved
src/include/common/types/types.h Outdated Show resolved Hide resolved
src/include/common/types/types.h Outdated Show resolved Hide resolved
src/include/function/cast/cast_functions.h Outdated Show resolved Hide resolved
@Ashleyhx Ashleyhx force-pushed the ashleyhx/complete-numeric-type-system branch from dd21ad2 to 4845cd4 Compare September 7, 2023 18:10
@Ashleyhx Ashleyhx marked this pull request as ready for review September 7, 2023 21:10
@codecov
Copy link

codecov bot commented Sep 8, 2023

Codecov Report

Patch coverage: 71.42% and project coverage change: -0.08% ⚠️

Comparison is base (134ced4) 90.03% compared to head (db80fff) 89.95%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1994      +/-   ##
==========================================
- Coverage   90.03%   89.95%   -0.08%     
==========================================
  Files         909      909              
  Lines       32928    32974      +46     
==========================================
+ Hits        29646    29663      +17     
- Misses       3282     3311      +29     
Files Changed Coverage Δ
src/common/vector/value_vector.cpp 92.92% <0.00%> (-1.60%) ⬇️
src/include/common/types/types.h 100.00% <ø> (ø)
...processor/operator/order_by/order_by_key_encoder.h 100.00% <ø> (ø)
src/include/storage/storage_structure/column.h 85.45% <ø> (ø)
...rc/include/storage/storage_structure/lists/lists.h 97.18% <ø> (ø)
src/storage/copier/column_chunk.cpp 75.71% <0.00%> (-0.74%) ⬇️
src/storage/copier/struct_column_chunk.cpp 32.47% <0.00%> (-0.28%) ⬇️
src/storage/storage_structure/in_mem_file.cpp 41.99% <ø> (ø)
src/storage/store/node_column.cpp 78.35% <ø> (ø)
src/storage/copier/table_copy_utils.cpp 75.82% <28.57%> (-1.01%) ⬇️
... and 11 more

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -274,6 +277,9 @@ void ArrowRowBatch::copyNonNullValue(
case LogicalTypeID::INT16: {
templateCopyNonNullValue<LogicalTypeID::INT16>(vector, typeInfo, value, pos);
} break;
case LogicalTypeID::INT8: {
templateCopyNonNullValue<LogicalTypeID::INT8>(vector, typeInfo, value, pos);
} break;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can remove the case statements in this file.

@@ -22,6 +22,8 @@ std::string PhysicalTypeUtils::physicalTypeToString(PhysicalTypeID physicalType)
return "INT32";
case PhysicalTypeID::INT16:
return "INT16";
case PhysicalTypeID::INT8:
return "INT8";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for exception handling, keep this.

@@ -502,6 +517,9 @@ void Value::serialize(FileInfo* fileInfo, uint64_t& offset) const {
case PhysicalTypeID::INT16: {
SerDeser::serializeValue(val.int16Val, fileInfo, offset);
} break;
case PhysicalTypeID::INT8: {
SerDeser::serializeValue(val.int8Val, fileInfo, offset);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep switch cases in this file, they are used for literalExpression in UDF. You don't need to add a test for this.

src/common/vector/value_vector.cpp Show resolved Hide resolved
src/include/common/types/value.h Show resolved Hide resolved
src/storage/copier/column_chunk.cpp Outdated Show resolved Hide resolved
src/storage/copier/npy_reader.cpp Outdated Show resolved Hide resolved
src/storage/copier/struct_column_chunk.cpp Show resolved Hide resolved
src/storage/copier/table_copy_utils.cpp Outdated Show resolved Hide resolved
src/storage/copier/table_copy_utils.cpp Show resolved Hide resolved
@Ashleyhx Ashleyhx force-pushed the ashleyhx/complete-numeric-type-system branch 2 times, most recently from 84b1c33 to 6ed50fb Compare September 11, 2023 21:56
Copy link
Collaborator

@acquamarin acquamarin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember to increase the KUZU_VERSION in CMAKE_LISTS.TXT, and src/include/storage/storage_info.h.

package.json Outdated Show resolved Hide resolved
package-lock.json Outdated Show resolved Hide resolved
src/include/common/type_utils.h Show resolved Hide resolved
src/include/common/types/types.h Outdated Show resolved Hide resolved
test/graph_test/graph_test.cpp Outdated Show resolved Hide resolved
@Ashleyhx Ashleyhx force-pushed the ashleyhx/complete-numeric-type-system branch from e43eea7 to 483dfa3 Compare September 12, 2023 00:39
@Ashleyhx Ashleyhx force-pushed the ashleyhx/complete-numeric-type-system branch from 483dfa3 to db80fff Compare September 12, 2023 02:02
@acquamarin acquamarin merged commit c80cf58 into master Sep 12, 2023
9 of 10 checks passed
@acquamarin acquamarin deleted the ashleyhx/complete-numeric-type-system branch September 12, 2023 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants