This is a bitstream realization for bits, whereas usual BitStreams can work only with bytes and are tough to understand how to be used. This is finestream!
You need to serialize data for streaming? Or write a Huffman Coding realization? Or some other encryption? FineStream will help you. Or maybe you need to write vector <bool>
to a file or to send it easily without wasting your time to understand vector <bool>
specific realization? FineStream comes in handy. You can compress your file up to 8x times for bool
s. Or - let's imagine - you need to store short integers from 0 to 15, do you REALLY need 8 bits? Or maybe 4 is enough? Then you could compress your file by 2x times, using FineStream with no effort.
Here's an example of data format created with finestream (.mmd18):
https://github.com/redmms/mercury_chess
At the moment FineStream can write bits to a file, read bits from a file and convert your data to a container of bytes. If the project gets some stars, new methods will be added, including built-in stream redirection not only to a file etc., everything depends on you.
You can easily compress your data with bitset
of random size, some container with bool
s or using my type, created specially for storing bit sequences not bigger than 1 byte - struct bitremedy
with lots of useful methods. FineStream will write them to a file (or read from) as bits without leading zeros and without casting to char
/string
.
You can also use any other type or container you want, it will not crash FineStream, FineStream will iterate all the elements itself, simply use <<
or >>
operator - no more need for boilerplate code. FineStream will work fine with every type and container, including union
s, struct
ures and enum
s, though it will not compress them as much as bool
s or bitset
s. Even container adaptors, which are: stack
, queue
and priority_queue
are not exception, ofinestream will make a copy of them to output, and will not change them.
Maximum speed at the moment is the speed of reading short bitset
s from a file, for bitset <3>
it is 27.7 MB/s with /O2 MSVC compiler argument (maximum optimization of speed). For bool
s it works only 15% slower than a usual fstream
, but writes 8x times more information per one byte.
It is also ready for running on embedded devices and other nonstandard machines with different byte sizes and different endianness.
There's one bitremedy
structure and one finestream
class with its children classes - ofinestream
for writing to a file and ifinestream
for reading from a file. They are stored in "module/bitremedy.ixx" and "module/finestream.ixx" files.
Use "module/example.cppm" to see how it works, just disallow double-slash comments where you need.
Use Windows binary editor to watch the result file and Windows programmer calculator to convert hex into binary format, or some analog of them.
unsigned char
UCBYTE is used to store from 0 to 8 (orCHAR_BIT
) bits.int
BITSN is used to store the number of bits you use. It's made int for convenience and less number of implicit type casts.bool
MOVED_LEFT is used to store info about if bit sequence is left aligned or not, true when left aligned and false when right aligned.
bitremedy
methods are:
In constructor you can initializebitremedy
CBYTE either with bitset <8> or char or type that can be automaticaly converted to char. Also you can use figure brackets for initialization.
ClearMargins()
is the method cleaning unused bits to erase all extra 1 bits. This method is called in all constructors, so when you initialize abitremedy
you can be sure it's alright, but you need to be careful when you change the value, or to useIsValid()
/ValidityTest()
methods if you are not sure.MoveToLeft()
andMoveToRight()
shift bit sequence close to the left or right edge respectively.AddToRight()
method add anotherbitremedy
passed as an argument to initial one, moving them all to the left side and returning newbitremedy
if their whole length is more then one byte size (orbitremedy
with zeros if no).Clear()
assigns zeros to allbitremedy
members, andClearToLeft()
assigns all zeros but true for MOVED_LEFT.- Other useful methods:
--AddToLeft/Right
--CopyNFromLeft/Right
--CutNFromLeft/Right
--RestoreLastAlign
--ToStr
- That's how you initialize a
bitremedy
:
All unused bits will be assigned 0 during initialization. But if abitremedy
passed to finestream will have nonempty margins it will throw an error.
NOTE: the project is updated regularly, so the pictures can be out of date a little bit.
- Install last version of MS Visual Studio
- Open Visual Studio.
- In Get Started menu press "Open project or solution" and choose .sln file from the repo
To compile it in other IDEs enable C++20 and for MSVC compiler use this compiler command: /dxifcInlineFunctions- if you have this MSVC bug:
https://developercommunity.visualstudio.com/t/Modules-ICE-when-using-cout-inside-of/10299789
NOTE:
In MS Visual Studio you may encounter this bug:
https://developercommunity.visualstudio.com/t/macros-vc-includepath-and-vc-librarypath-x86-expan/4974
Try to add ucrt library paths manually to the project properties:
https://stackoverflow.com/questions/33104562/error-c1083-cannot-open-include-file-crtdbg-h-no-such-file-or-directory/77377130#77377130
https://developercommunity.visualstudio.com/t/ucrt-doesnt-work-in-x64-msbuild/1184283#T-N1201257
- Open "example.cppm" file.
- Uncomment an example you need:
- Watch the information about the last byte. Here 6 is the length of FineStream and 2 is the number of the last byte's unused bits:
- Add recently created "output.txt" to your project.
- Open it with Windows Binary Editor:
- Watch the result:
- Use Windows calculator in programmer regime to know exactly how your bits sequence looks like:
You can use any type with finestream
, though tuples are least tested at the moment.
For example, here are random size containers with ofinestream
:
fn::ofinestream bsm("../output.txt");
vector<vector<uint8_t>> tree {
{ 0, 1, 3 },
{ 4, 3 },
{ 1, 3, 5, 7, 9 }
};
bsm << tree;
And here with ifinestream
:
fn::ifinestream bsm("../output.txt");
vector<vector<uint8_t>> read_tree{
{ 0, 0, 0 },
{ 0, 0 },
{ 0, 0, 0, 0, 0 }
};
bsm >> read_tree;
Another way to do that:
fn::ifinestream bsm("../output.txt");
vector<vector<uint8_t>> read_tree(3);
read_tree[0].resize(3);
read_tree[1].resize(2);
read_tree[2].resize(5);
bsm >> read_tree;
fn::ofinestream bsm("../output.txt");
queue<uint8_t> qout({ 1, 2, 3, 4, 5, 6 });
bsm << qout; // qout will still contain that 6 elements
To input data just replace 'o' by 'i' in fn::ofinestream:
fn::ifinestream bsm("../output.txt");
queue<uint8_t> qin({ 0, 0, 0, 0, 0, 0 });
bsm >> qin; // now qin == qout
while (!qin.empty()) {
cout << (int)qin.front() << endl;
qin.pop();
}
Output:
1
2
3
4
5
6
tuple<int, double, char> tup{ 1, 3.14, 'a' }; // includes 2 bytes of garbage
bsm << tup;
Hexadecimal output:
00 00 00 00 00 00 00 01 // some bytes here are garbage and may be not 00
40 09 1E B8 51 EB 85 1F
00 00 00 00 00 00 03 61 // 03 is not your data
Warning in cerr:
WARNING: are you sure about this type - class std::tuple<int,double,char>?
tuple<long long, double, uint32_t, char, char, char, char> lltup{ 1, 3.14, 32, 'a', 'a', 'a', 'a'};
// no garbage, only leading zeros
bsm << lltup;
Hexadecimal output:
00 00 00 00 00 00 00 01
40 09 1E B8 51 EB 85 1F
00 00 00 20 61 61 61 61
uint8_t i1 = 0b10; // i1 == 2
vector<bool> v1 = fn::ToVector(i1); // v1.size() == 8, v1[0] == false, v1[1] == true
uint8_t i2 = fn::FromVector<uint8_t>(v1); // i2 == 2
vector<bool> v2 = fn::ToSizedVector(i1, 2); // v2.size() == 2, v2[0] == false, v2[1] == true
uint8_t i3 = fn::FromVector<uint8_t>(v2); //i3 == 2
uint8_t u8;
cout << "uint8_t: " << fn::BitSize(u8) << endl;
bitset<98> bs98;
cout << "bitset<98>: " << fn::BitSize(bs98) << endl;
forward_list<uint8_t> fl9(9);
cout << "forward_list<uint8_t>: " << fn::BitSize(fl9) << endl;
uint16_t arr3[3];
cout << "uint16_t [3]: " << fn::BitSize(arr3) << endl;
Output:
uint8_t: 8
bitset<98>: 98
forward_list<uint8_t>: 72
uint16_t [3]: 48
vector<vector<uint8_t>> empty_tree{
{ 0, 0, 0 },
{ 0, 0 },
{ 0, 0, 0, 0, 0 }
};
empty_tree[0][1] = 0b0100'0000;
cout << fn::IntNonLeadingN(empty_tree) << endl; // 71
bitset<11> bstt(0);
bstt[1] = 1;
cout << bstt << endl; // 00000000010
cout << fn::IntLeadingN(bstt) << endl; // 9