Release v1.7.1.1

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

Deploy

• upgrade mysql to version 8.0.28

Eggroll

• Support Eggroll v2.4.3, upgrade com.h2database:h2 to version 2.1.210, com.google.protobuf:protobuf-java to version 3.16.1

Release v1.7.1

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FederatedML

• Iterative Affine is disabled
• Speed up Hetero Feature Selection, 100x+ faster when feature dimension is high
• Speed up OneHot, 60x+ faster when feature dimension is high
• Data Statistics supports missing value, with improved efficiency
• Fix bug of quantile binning: may lose data when partitions hold too many instances
• Fix reconstruction reuse problem of SPDZ
• Fix Host's ineffective decay rate of Homo Logistic Regression
• Improved strategy for handling missing values when converting Homo SecureBoost using homo model convertor
• Improved presentation of Evaluation's confusing matrix

FATE-Client

• Add Source Provider attribute to Pipeline components

Eggroll

• Support Eggroll v2.4.2, fixed Log4j security bug

Release v1.7.0

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FATE-ARCH

• Support EggRoll 2.4.0
• Support Spark-Local Computing Engine
• Support Hive Storage
• Support LocalFS Storage for Spark-Local Computing Engine
• Optimizing the API interface for Storage session and table
• Simplified the API interface for Session, remove backend and workmode parameters
• Heterogeneous Engine Support: Federation between Spark-Local and Spark-Cluster
• Computing Engine, Storage Engine, Federation Engine are set in conf/service_conf.yaml when FATE is deployed

FederatedML

• Optimized Hetero-SecureBoost: with gradient packing、cipher compressing, and sparse point statistics optimization, 4x+ faster
• Homo-SecureBoost supports memory-based histogram computation for more efficient tree building, 5x+ faster
• Optimized RSA Intersect with CRT optimization, 3x+ faster
• New two-party Hetero Logistic Regression Algorithm: mixed protocol of HE and MPC, without a trusted third party
• Support data with match-id, separating match id and sample id
• New DH Intersect based on PH Key-exchange protocol
• Intersect support cardinality estimation
• Intersect adds optionally preprocessing step
• RSA and DH Intersect support cache
• New Feature Imputation module: can apply arbitrary imputation method to each column
• New Label Transform module: transform categorical label values
• Homo-LR, Homo-SecureBoost, Homo-NN now can convert models into sklearn、lightgbm、torch & tf-keras framework
• Hetero Feature Binning supports multi-class task, higher efficiency with label packing
• Hetero Feature Selection support multi-class iv filter
• Secure Information Retrieval supports multi-column retrieval
• Major training algorithms support warm-start and checkpoint : Homo & Hetero LR, Homo & Hetero-SecureBoost, Homo & Hetero NN
• Optimized Pailler addition operation, several times faster, Hetero-SecureBoost with Paillier speed up 2x+

Fate-Client

• Pipeline supports uploading match id functionality
• Pipeline supports homo model conversion
• Pipeline supports model push to FATE-Serving
• Pipeline supports running jobs with specified FATE version

FATE-Test

• Integrate FederatedML unittest
• Support for uploading image data
• Big data generation using storage interface, optimized generation logic
• Support for historical data comparison
• cache_deps and model_loader_deps support
• Run DSL Testsuite with specified FATE version

Release v1.6.1

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FederatedML

• Support single party prediction
• SIR support non-ascii id
• Selection support local iv filter
• Adjustable Paillier key length for Hetero LR
• Binning support iv calculation on categorical features
• Hetero LR one vs rest support evaluation during training

FATE-Flow:

• Support mysql storage engine;
• Added service registry interface;
• Added service query interface;
• Support fate on WeDataSphere mode
• Add lock when writing model_local_cache
• Register the model download urls to zookeeper

Bug-Fix:

• Fix error for deploying module with lack of partial upstream modules in multi-input cases
• Fix error for deploying module with multiple output, like data-statistics
• Fix job id length no more than 25 limitation
• Fix error when loss function of Hetero SecureBoost set to log-cosh
• Fix setting predict label to string-type error when Hetero SecureBoost predicts
• Fix error for HeteroLR without intercept
• Fix quantile error of Data Statistics with specified columns
• Fix string parsing error of OneHot with specified columns
• Some parameters can now take 0 or 1 integer values when valid range is [0, 1]

Release v1.5.2

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FederatedML

• SIR support non-ascii id
• Selection support local iv filter
• Adjustable Paillier key length for Hetero LR
• Binning support iv calculation on categorical features
• Hetero LR one vs rest support evaluation during training

Fate-Flow

• Read data from mysql with ‘table bind’ command to map source table to FATE table
• FATE cluster push model for one-to-multiple FATE Serving clusters in one party

System Architecture

• More efficient ‘sample’ api

Bug Fixes

• Fix error for deploying module with lack of partial upstream modules in multi-input cases
• Fix job id length no more than 25 limitation
• Fix error when loss function of Hetero SecureBoost set to log-cosh
• Fix setting predict label to string-type error when Hetero SecureBoost predicts
• Fix error for HeteroLR without intercept
• Fix torch import error
• Fix quantile error of Data Statistics with specified columns
• Fix string parsing error of OneHot with specified columns
• Some parameters can now take 0 or 1 integer values when valid range is [0, 1]

Release v1.6.0

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FederatedML

• Hetero SecureBoost: more efficient computation with GOSS, histogram subtraction, cipher compression, 2-4x faster
• Hetero GLM: improved communication efficiency, adjustable floating point precision, 2x faster
• Hetero NN: adjustable floating point precision, support SelectiveBackPropagation and dropOut on interaction layer, 2x faster
• Hetero Feature Binning: improved algorithm with cipher compression, 2x faster
• Intersect: add split calculation option and adjustable random base fraction, 30% faster
• Homo NN: restructure torch backend and enhanced grammar; train and predict with raw image data
• Intersect supports SM3 hashing method
• Hetero SecureBoost: L1 penalty & adjustable min_child_weight to prevent overfitting
• NEW SecureBoost Transformer: feature engineering module that encodes instances with leaf nodes from SecureBoost model
• Hetero Pearson: support local VIF computation
• Hetero Feature Selection: support selection based on VIF and Pearson
• NEW Homo Feature Binning: support virtual/recursive binning strategy
• NEW Sample Weight: set sample weights based on label or from feature column, Hetero GLM & Hetero SecureBoost support weighted training
• NEW Data Transformer: case-insensitive on data schema
• Local Baseline supports prediction task
• Cross Validation: output fold split history
• Evaluation: add multi-result-unfold option which unfolds multi-classification evaluation result to several binary evaluation results in a one-vs-rest manner

System Architecture

• Added local file system directory path virtual storage engine to support image input data
• Added the message queue Pulsar cross-site transmission engine, which can be used with the Spark computing engine, and can be added to the Exchange role to support the star networking mode

FATE-Test

• Add Benchmark performance for efficiency comparison; add mock data generation tool; support metrics comparison between training and validation sets
• FATE-Flow unittest for REST/CLI/SDK API and training-prediction workflow

Release v1.5.1

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FederatedML

• Add Feldman Verifiable Secret Sharing protocol (contributed)
• Add Feldman Verifiable Sum Module (contributed)
• Updated FATE-Client and FATE-Test for new FATE-Flow
• Upgraded early stopping strategy: record best model for each metric

Fate-Flow

• Optimize the model center, reconstruct publishing model, support deploy, load, bind, migrate operations, and add new interfaces such as model info
• Improve identity authentication and resource authorization, support party identity verification, and participate in the authorization of roles and components
• Optimize and fix resource manager, add task_cores job parameters to adapt to different computing engines

Eggroll

• In one-way communication mode, add party identity authentication function, which needs to be used with FATE-Cloud

Deploy

• Support 1.5.0 retain data upgrade to 1.5.1

Bug Fixes

• Fix predict-cache in SecureBoost validation
• Fix job clean CLI

Release v1.5.0（LTS）

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Release 1.5.0（LTS）

Major Features and Improvements

FederatedML

• Refactored Hetero FTL with optional communication-efficiency mechanism, with 4x time efficiency improvement
• Hetero SecureBoost supports complete secure mode
• Hetero SecureBoost now can reduce time consumption over highly sparse data by using sparse matrix
  computation on histogram aggregations.
• Hetero SecureBoost optimization: the communication round in prediction is reduced to no larger than tree depth,
  prediction speed is improved by 32 times in a 100-tree model.
• Addition of Hetero FastSecureBoost module, whose mixed/layered modeling method makes it twice as efficient as SecureBoost
• Improved Hetero Federated Binning with 30%~50% time efficiency improvement
• Better GLM: >10% improvement in time efficiency
• FATE first unsupervised learning algorithm: Hetero KMeans
• Upgraded Hetero Feature Selection: add PSI filter and SecureBoost feature importance filter
• Add Data Split module: splitting data into train, validate, and test sets inside FATE modeling workflow
• Add DataStatistic module: compute min/max, mean, median, skewness, kurtosis, coefficient of variance, percentile, etc.
• Add PSI module for computing population stability index
• Add Homo OneHot module for one-hot encoding in homogeneous scenario
• Evaluation module adds metrics for clustering
• Optional FedProx mechanism for Homo LR, useful for training with non-iid data
• Add Oblivious Transfer Protocol and OT-based module Secure Information Retrieval
• Random Iterative Affine protocol, providing additional security

Fate-Flow

• Brand new scheduling framework based on global state and optimistic concurrency control and support multiple scheduler
• Upgraded task scheduling: multi-model output for component, executing component in parallel, component rerun
• Add new DSL v2 which significantly improves user experiences in comparison to DSL v1. Several syntax error detection functions are supported in v2. Now DSL v1 and v2 are
  compatible in the current FATE version
• Enhanced resource scheduling: remove limit on job number, base on cores, memory and working node according to different computing engine supports
• Add model registry, supports model query, import/export, model transfer between clusters
• Add Reader component: automatically dump input data to FATE-compatible format and cluster storage engine; now data from HDFS
• Refactor submit job configuration's parameters setting, support different parties use different job parameters when using dsl V2.

System Architecture

• New architectural framework that supports a combination of different computing, storage, and transfer engines
• Support new engine combination: Spark、HDFS、RabbitMQ
• New data table management, standardized API for all different storage engines
• Rearrange FATE code structure, conf setting at one place, streamlined user experiment
• Support one-way network communication between parties, only one party needs to open the entrance network strategy

FATE-Client

• Pipeline, a tool with a keras-like user interface and integrates TensorFlow, PyTorch, Keras in the backend, is used for fast federated model building with FATE
• Brand new CLI v2 with easy independent installation, user-friendly programming syntax & command-line prompt
• Support FLOW python language SDK
• Support PyPI

FATE-Test

• Testsuite: For Fate function regressions
• Benchmark tool and examples for comparing modeling quality; provided examples include common models such as heterogeneous LR, SecureBoost, and NN
• Performance Statistics: Log now includes statistics on timing, API usage, and variable transfer

Release v1.4.6

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FederatedML

• Add Column Expand Module
• Add Scorecard Module

Release v1.5.0（LTS）-preview

By downloading, installing or using the software, you accept and agree to be bound by all of the terms and conditions of the LICENSE and DISCLAIMER.

Major Features and Improvements

FederatedML

• Refactored Hetero FTL with optional communication-efficiency mechanism, with 4x time efficiency improvement
• Hetero SecureBoost supports complete secure mode
• Hetero SecureBoost now can reduce time consumption over highly sparse data by using sparse matrix
  computation on histogram aggregations.
• Hetero SecureBoost optimization: the communication round in prediction is reduced to no larger than tree depth,
  prediction speed is improved by 32 times in a 100-tree model.
• Addition of Hetero FastSecureBoost module, whose mixed/layered modeling method makes it twice as efficient as SecureBoost
• Improved Hetero Federated Binning with 30%~50% time efficiency improvement
• Better GLM: >10% improvement in time efficiency
• FATE first unsupervised learning algorithm: Hetero KMeans
• Upgraded Hetero Feature Selection: add PSI filter and SecureBoost feature importance filter
• Add Data Split module: splitting data into train, validate, and test sets inside FATE modeling workflow
• Add DataStatistic module: compute min/max, mean, median, skewness, kurtosis, coefficient of variance, percentile, etc.
• Add PSI module for computing population stability index
• Add Homo OneHot module for one-hot encoding in homogeneous scenario
• Evaluation module adds metrics for clustering
• Optional FedProx mechanism for Homo LR, useful for training with non-iid data
• Add Oblivious Transfer Protocol and OT-based module Secure Information Retrieval
• Random Iterative Affine protocol, providing additional security

Fate-Flow

• Brand new scheduling framework based on global state and optimistic concurrency control and support multiple scheduler
• Upgraded task scheduling: multi-model output for component, executing component in parallel, component rerun
• Add new DSL v2 which significantly improves user experiences in comparison to DSL v1. Several syntax error detection functions are supported in v2. Now DSL v1 and v2 are
  compatible in the current FATE version
• Enhanced resource scheduling: remove limit on job number, base on cores, memory and working node according to different computing engine supports
• Add model registry, supports model query, import/export, model transfer between clusters
• Add Reader component: automatically dump input data to FATE-compatible format and cluster storage engine; now data from HDFS

System Architecture

• New architectural framework that supports a combination of different computing, storage, and transfer engines
• Support new engine combination: Spark、HDFS、RabbitMQ
• New data table management, standardized API for all different storage engines
• Rearrange FATE code structure, conf setting at one place, streamlined user experiment

FATE-Client

• Pipeline, a tool with a keras-like user interface and integrates TensorFlow, PyTorch, Keras in the backend, is used for fast federated model building with FATE
• Brand new CLI v2 with easy independent installation, user-friendly programming syntax & command-line prompt
• Support FLOW python language SDK
• Support PyPI

FATE-Test

• Testsuite: For Fate function regressions
• Benchmark tool and examples for comparing modeling quality; provided examples include common models such as heterogeneous LR, SecureBoost, and NN
• Performance Statistics: Log now includes statistics on timing, API usage, and variable transfer