Applied Sciences, 2024, 14, 2864
This paper presents a comprehensive study of Local Differential Privacy (LDP) schemes aimed at safeguarding sensitive data in healthcare applications. While LDP is increasingly vital for protecting privacy when handling high-dimensional and highly correlated medical data, standard approaches struggle with the curse of dimensionality. We introduce Bayesian Ridge Regression (BRR) for joint probability estimation, comparing its performance to two baseline methods (LOPUB and LOCOP) on 15 real-world datasets with varying average absolute Pearson correlation among attributes. Our results show that BRR consistently achieves lower error across different privacy budgets, especially in datasets with weaker correlations between features. This underlines the importance of model selection in LDP, offering a path forward for robust privacy solutions in healthcare and beyond.
The exponential growth of healthcare technology and digital services has ignited pressing data-privacy concerns. Healthcare providers routinely collect massive patient records, including personally identifiable medical data. Striking a balance between maximizing utility for analytics and preserving privacy is crucial. One promising methodology is Local Differential Privacy (LDP), which encodes and perturbs user data before transmission to a central server.
Nonetheless, typical LDP methods face inherent scalability issues known as the curse of dimensionality,
especially problematic for k-way joint probability distribution estimation (where increasing numbers of
attributes demand significant noise injection). Two noteworthy solutions, LOPUB and
LOCOP, rely on LASSO regression to reconstruct distributions from noisy Bloom-filtered data.
However, their accuracy degrades for datasets featuring strongly correlated variables or high-dimensional
contexts.
In this work, we propose integrating Bayesian Ridge Regression (BRR) into standard LDP pipelines to mitigate these problems. BRR offers an advantage by handling multicollinearity more effectively, preventing the over-penalization or elimination of strongly associated variables (a limitation of LASSO). Our experiments focus heavily on healthcare data, characterized by intricate correlations (e.g., body mass index vs. disease risk).
LDP addresses privacy at the user side: each user applies noise to their data before sending it to
the central server. Formally, a mechanism ξ satisfies
ε-local differential privacy if, for any two data points
u and w, and any output ξ(u),
Pr[ξ(u) = r] ≤ exp(ε) * Pr[ξ(w) = r].
Higher privacy requires more noise, aggravating the curse of dimensionality in multi-attribute data.
As the number of features (e.g., patient attributes) grows, LDP demands injecting more noise to keep
privacy levels fixed. This noise injection rapidly deteriorates utility, impeding accurate estimation
of high-dimensional or heavily correlated attributes. Approaches like LOPUB and
LOCOP have attempted to reduce overhead by using Bloom filters and LASSO, but their
performance declines substantially when the attribute count grows large or when correlation among
attributes is high.
We build upon the LOPUB scheme, where each user encodes each attribute into a Bloom filter, applies
random bit-flipping for privacy, and sends the noisy result. The server aggregates the data and
computes a candidate bit matrix M, along with a “noisy count” vector y.
Instead of LASSO, we apply BRR to solve M = βy + ϑ. BRR introduces a Gaussian prior on the
parameters β and employs Bayesian inference, which better handles multicollinearity
(i.e., when some features are linearly dependent).
In classical Ridge Regression, we minimize the sum of squared residuals plus a
L2-norm penalty:
L = (1/2n) ||M - βy||^2 + α||β||^2.
BRR incorporates a Bayesian perspective, interpreting β as normally distributed around 0.
This differs from LASSO’s Laplacian prior, which tends to drive some coefficients to 0 and can be less
stable for highly correlated features. BRR yields smooth and stable coefficient estimates, often
advantageous in healthcare datasets where attributes are strongly correlated (e.g., disease symptoms).
Each user’s local randomization (via Bloom filters + randomized response) satisfies
ε·d LDP for d attributes. The sequential composition theorem applies since
each attribute is perturbed independently. Our use of BRR on the server side does not compromise
local privacy guarantees.
We test on 15 publicly available datasets, with 7 from healthcare (e.g. Stroke, Diabetes, NHANES, etc.) and 8 from finance, social science, and more. Continuous features are discretized, and each attribute is mapped via a label encoder to maintain consistent cardinalities. We measure average absolute correlation among attributes (AAR), a key factor for LDP performance.
AVD vs. Privacy Budget: For each dataset, with k ∈ [2, 5] joint distributions
under privacy budgets ε ∈ [0.1, 1.0], BRR consistently yields lower AVD than LOPUB (LASSO)
or LOCOP (Copula + LASSO). Gains are especially large for smaller ε (strong privacy).
R2 Score: BRR attains higher R-squared. This corroborates that Bayesian priors help in correlated environments, preventing over-penalization or elimination of key attributes.
Sensitivity to AAR: As AAR rises (more correlated attributes), performance of all methods degrades, but BRR remains significantly more stable than LOPUB or LOCOP. The differences are modest when the correlation is extreme, but in moderately correlated datasets, BRR’s advantage becomes substantial.
Our findings highlight how the strong correlation of attributes in healthcare (e.g.,
disease profiles, blood markers) intensifies noise propagation under LDP. While LOPUB
and LOCOP degrade quickly in accuracy, Bayesian Ridge Regression is more resilient,
achieving lower reconstruction error across varied correlation levels.
The direct advantage is enabling healthcare analysts and epidemiologists to study multi-attribute
distributions—for example, disease prevalence across demographic subgroups—without unacceptable
privacy/utility trade-offs. However, dimensionality constraints persist: at very large d,
LDP still faces fundamental challenges, and advanced dimensionality reduction or attribute selection
steps may be necessary. We also emphasize the need to monitor AAR during data publishing, possibly
restricting heavily correlated attributes or employing advanced correlation-breaking transformations
for higher security.
We introduced a novel Local Differential Privacy scheme centered on Bayesian Ridge Regression (BRR) for high-dimensional, correlated datasets, with an emphasis on healthcare. This approach outperforms existing LDP solutions such as LOPUB and LOCOP when evaluating multi-attribute joint probability distributions at various privacy budgets and dimension settings.
By mitigating the pitfalls of LASSO, BRR delivers higher accuracy and stability, demonstrating better resilience to strong feature correlations. For future work, we plan to investigate ensembles of Bayesian priors, advanced dimensionality reduction, and expansions to dynamic or longitudinal patient data. Our results are promising for the practical deployment of LDP in complex real-world healthcare scenarios, promoting both innovation and rigorous patient privacy.