Privacy-Preserving CNN Training with Transfer Learning: Multiclass Logistic Regression

AmazUtah_NLP at SemEval-2024 Task 9: A MultiChoice Question Answering System for Commonsense Defying Reasoning


View a PDF of the paper titled Privacy-Preserving CNN Training with Transfer Learning: Multiclass Logistic Regression, by John Chiang

View PDF

Abstract:In this paper, we present a practical solution to implement privacy-preserving CNN training based on mere Homomorphic Encryption (HE) technique. To our best knowledge, this is the first attempt successfully to crack this nut and no work ever before has achieved this goal. Several techniques combine to accomplish the task:: (1) with transfer learning, privacy-preserving CNN training can be reduced to homomorphic neural network training, or even multiclass logistic regression (MLR) training; (2) via a faster gradient variant called $texttt{Quadratic Gradient}$, an enhanced gradient method for MLR with a state-of-the-art performance in convergence speed is applied in this work to achieve high performance; (3) we employ the thought of transformation in mathematics to transform approximating Softmax function in the encryption domain to the approximation of the Sigmoid function. A new type of loss function termed $texttt{Squared Likelihood Error}$ has been developed alongside to align with this change.; and (4) we use a simple but flexible matrix-encoding method named $texttt{Volley Revolver}$ to manage the data flow in the ciphertexts, which is the key factor to complete the whole homomorphic CNN training. The complete, runnable C++ code to implement our work can be found at: href{this https URL}{$texttt{this https URL}$}. We select $texttt{REGNET_X_400MF}$ as our pre-trained model for transfer learning. We use the first 128 MNIST training images as training data and the whole MNIST testing dataset as the testing data. The client only needs to upload 6 ciphertexts to the cloud and it takes $sim 21$ mins to perform 2 iterations on a cloud with 64 vCPUs, resulting in a precision of $21.49%$.

Submission history

From: John Chiang [view email]
[v1]
Fri, 7 Apr 2023 18:21:30 UTC (300 KB)
[v2]
Tue, 9 May 2023 20:51:07 UTC (300 KB)
[v3]
Tue, 4 Jun 2024 01:18:21 UTC (187 KB)
[v4]
Sun, 12 Jan 2025 03:49:50 UTC (187 KB)



Source link
lol

By stp2y

Leave a Reply

Your email address will not be published. Required fields are marked *

No widgets found. Go to Widget page and add the widget in Offcanvas Sidebar Widget Area.