Comparison of the performance of three cardiovascular risk assessment models in identifying vulnerable carotid plaques

Objective To evaluate the applicability of three cardiovascular risk assessment models as primary screening tools for vulnerable carotid plaques.

Methods Using a cross-sectional design, we enrolled 986 local residents from Liuzhou, Guangxi surveyed between 2019 and 2023 as participants. The 10-year cardiovascular disease risk values were calculated using the assessment models of prediction for atherosclerotic cardiovascular disease risk in China (China-PAR), Framingham risk score (FRS), and World Health Organization cardiovascular disease risk charts (WHO laboratory-based model). The model performance was assessed using the area under the receiver operating characteristic curve (AUC), Hosmer-Lemeshow goodness-of-fit test, calibration plots, sensitivity, specificity and positive predictive value, and the optimal cut-offs for each model were explored by maximizing the Youden's index.

Results The 3 models exhibited good discriminatory ability for vulnerable carotid plaques (AUC: 0.71–0.72), but showed poor calibration (P < 0.001). At the original cut-offs of each model, the FRS showed high sensitivity (89.6%), low specificity (32.2%) and low positive predictive value (27.6%). The other two models showed low sensitivity (WHO laboratory-based model: 39.8%; China-PAR model: 53.4%), high specificity (WHO laboratory-based model: 84.2%; China-PAR model: 72.5%), and high positive predictive value (WHO laboratory-based model: 42.1%; China-PAR model: 35.9%). After cut-off optimization, the WHO laboratory-based model at the optimal cut-off of 4.5% exhibited a higher sensitivity than the FRS model (74.2% vs. 67.0%, P=0.003) and the China-PAR model (74.2% vs. 67.9%, P=0.018), and exhibited a lower specificity than the China-PAR model (61.7% vs. 65.2%, P=0.006) and the FRS model (61.7% vs. 66.5%, P < 0.001). No statistically significant differences in positive predictive value were observed between the WHO laboratory-based model (35.9%) and either the China-PAR model (36.1%) or the FRS model (36.6%)(P > 0.05).

Conclusion The 3 models exhibit satisfactory discriminatory performance in identifying vulnerable carotid plaques. However, when directly applying the original development cut-offs for initial screening, it is difficult to balance missed diagnoses and misdiagnoses. After cut-off optimization, the WHO laboratory-based model achieves the best overall performance. These conclusions suggest that primary medical institutions in Guangxi region can initially use the cut-off risk (4.5%) optimized by the WHO laboratory-based model for initial screening and conduct further carotid ultrasonography for high-risk individuals.