Viadot 0.4.22
What's Changed
Added
- Added
TM1
source class. - Added
TM1ToDF
task class. - Added
set_prefect_kv
parameter toBigQueryToADLS
withFalse
as a default. If there is a need to create new pair in KV Store the parameter can be changed toTrue
. - Added
_rename_duplicated_fields
method toSharepointListToDF
task class for finding and rename duplicated columns. - Added new view type
agent_interaction_view_type
inGenesys
source. - Added new logic for endpoint
users
inGenesys
task. - Added libraries
nltk
andsklearn
torequirements
.
Fixed
- Fixed bug for endpoint
conversations
in GET method inGenesys
Task.
Changed
- Splitted test for
Eurostat
on source tests and task tests. - Modified
SharepointList
source class:
-> Docstrings update. - Modified
SharepointToADLS
flow class:
-> Docstrings update.
-> Changed set_prefect_kv: bool = False to prevent forced KV store append. - Modified
SharepointListToADLS
flow class:
-> Changed set_prefect_kv: bool = False to prevent forced KV store append. - Modified
SharepointList
source class:
-> Docstrings update.
-> Changed_unpack_fields
method to handle Sharepoint MultiChoiceField type + small improvements.
-> Changedget_fields
method to handle special characters - different approach to call get() and execute_query().
-> Renamed method fromselect_expandable_user_fields
toselect_fields
+ update for MultiChoiceField type.
-> Changedcheck_filters
method errors messages and more checks added.
-> Changedoperators_mapping
method errors messages.
-> Changedmake_filter_for_df
method errors messages. - Modified
SharepointListToDF
task class:
-> Docstrings update - Splitted test for Eurostat on source tests and task tests.
- Modified
CustomerGauge
source class with simplified logic to return json structure. - Expanded
CustomerGaugeToDF
task class with separate cleaning functions and handling nested json structure flattening with two new methods_field_reference_unpacker
and_nested_dict_transformer
. - Changed
CustomerGaugeToADLS
to containing new arguments.
Shortcut
- Genesys agent interaction view type by @Diego-H-S in #803
- Added TM1 connector by @angelika233 in #801
- Eurostat tests improvments by @adrian-wojcik in #806
- ⚡️ Added
set_prefect_kv
parameter toBigQueryToADLS
flow by @Rafalz13 in #809 - Improve test coverage for viadot source package by @Rafalz13 in #807
- Dev customer gauge unpacker by @hhnks in #800
- ✨ Add new requirements by @malgorzatagwinner in #813
- Feature/sharepoint list extension kpi5 by @marcinpurtak in #810
- web_msg logic fix by @burzekj in #812
- Fixed typo in tests by @marcinpurtak in #814
- ✨ new logic to extracting users from genesys by @burzekj in #811
- Revert of changes for desired columns by @marcinpurtak in #815
- Release 0.4.22 PR by @Rafalz13 in #816
New Contributors
- @malgorzatagwinner made their first contribution in #813
Full Changelog: v0.4.21...v0.4.22