Lung Disease Detection | End-to-End Medical AI Platform

Artifact	Role	Used By
`all_metadata.csv`	Image-level metadata for 21,165 CXR records.	EDA, model training, MLOps evaluation.
`all_image_mask_pairs.csv`	Pairs each X-ray with its lung mask.	Segmentation, ROI cropping, mask-aware classification.
`class_mapping.json`	Original four-class mapping.	Research notebooks.
`healthy_binary_mapping.json`	Healthy/unhealthy output mapping.	Binary ensemble and API response labels.
`diseases_mapping.json`	COVID, Viral Pneumonia, Lung Opacity mapping.	Disease classifier and deployment assets.

EDA Question	What Was Checked	Why It Matters
Are masks available?	Image paths and mask paths are paired in a single metadata file.	The segmentation model and ROI-based classifiers require mask supervision.
Are class labels stable?	JSON mappings are generated for original four-class, binary, and disease-only tasks.	Prevents label mismatch between research, deployment, and UI labels.
Is the task split sensible?	Normal is preserved for binary triage, while unhealthy classes become disease classifier inputs.	Avoids asking the disease model to classify healthy scans as a disease.
Can the data stream repeatably?	TFRecord shards store image, mask, class label, and binary label.	Training and retraining jobs can consume the same artifact format.

Training Libraries First

Modeling Libraries & Training Dependencies

The training tab starts with dependencies because every notebook uses a slightly different mix of Keras backbones, mask processing, callbacks, metrics, and optimization tools.

TensorFlow / KerasTransfer-learning backbones, custom heads, metrics, callbacks, saved .keras artifacts, TFRecord datasets, and model export preparation.

Mask + ROI preprocessingEvery binary model uses the lung mask differently: ROI crop, mask channel, masked background, or mask-as-RGB input.

AUC, precision, recall, F1Model comparison is centered on F1 for classification, while detailed sections still show AUC, precision, recall, accuracy, threshold, and validation loss.

OptunaUsed for DenseNet binary and disease classifier architecture search, optimizer search, staged LR, dropout, label smoothing, and class-weight tuning.

YAML + JSON contractsFinal training decisions are saved into model metadata, mappings, Optuna JSON, and deployment model folders.

Training Strategy

Six-Model Research Program

The project trains binary triage models, a disease classifier, and segmentation models rather than relying on one monolithic classifier.

The research layer is split by clinical decision: first isolate lungs, then decide healthy versus unhealthy, then classify the disease if pathology is detected. Binary classification is intentionally ensemble-based: DenseNet121, EfficientNetV2B3, InceptionV3, and MobileNetV3 vote on the final healthy/unhealthy decision. A separate DenseNet121 model handles disease sub-classification for unhealthy scans.

Model	Task	Preprocessing	Threshold / Output	F1-First Metrics
`EfficientNetV2B3`	Binary healthy/unhealthy	4-channel RGB + mask, normalize [-1,1]	Threshold 0.85	F1 0.9544, AUC 0.9871, precision 0.9641, recall 0.9448
`DenseNet121`	Binary healthy/unhealthy	Lung ROI + DenseNet preprocessing	Threshold 0.91	F1 0.9245, AUC 0.9711, precision 0.9183, recall 0.9308
`MobileNetV3`	Binary healthy/unhealthy	Mask-as-RGB, masked input	Threshold 0.50	F1 0.9206, AUC 0.9730, precision 0.9259, recall 0.9153
`InceptionV3`	Binary healthy/unhealthy	Mask-as-RGB, background fill -1.0	Threshold 0.50	F1 0.9160, AUC 0.9699, precision 0.9244, recall 0.9077
`DenseNet121`	COVID / Viral Pneumonia / Lung Opacity	Lung ROI + DenseNet preprocessing	3-class softmax	F1 0.9782, AUC 0.9982, precision 0.9781, recall 0.9763
`U-Net Xception`	Lung segmentation	256x256 RGB input	Mask threshold 0.50	Accuracy 0.9885, Dice 0.9660, IoU 0.9533

Research Notebook Map

Training Subsections and Notebook Coverage

The training tab is split like a small research menu because the project contains several separate experiments, not one notebook.

Notebook / File Group	Research Role	Important Details Preserved in Page
`MobileNetV3-healthy-binary_classification.ipynb`	Lightweight binary classifier	Mask-as-RGB preprocessing, threshold 0.50, F1 0.9206, AUC 0.9730.
`InceptionV3-healthy-binary_classification.ipynb`	Binary classifier with Inception features	Masked background fill, threshold 0.50, F1 0.9160, AUC 0.9699.
`EfficientNetV2B3-healthy-binary_classification.ipynb`	Strongest binary model	RGB plus mask channel input, threshold 0.85, best binary F1 0.9544, AUC 0.9871.
`DenseNet121-healthy_binary_classification.ipynb`	ROI-based binary model	DenseNet preprocessing, ROI crop, conservative threshold 0.91, F1 0.9245.
`DenseNet121-healthy_binary_optuna_phase1/2.ipynb`	Binary hyperparameter search	Classifier head 1024/512, AdamW, cosine decay, dropout, label smoothing, staged LR.
`DenseNet121-diseases-multiclass_classification.ipynb`	Disease sub-classifier	Three-class unhealthy diagnosis with AUC 0.9982 and F1 0.9782.
`DenseNet121-diseases_optuna_phase1/2.ipynb`	Disease optimization	Compact 256/64 head, class-weight power, label smoothing, dropout 0.45.
`metadata.yaml` files	Deployment contract	Thresholds, class labels, preprocessing, artifact paths, ONNX paths, and reported metrics.

Tabbed Model Training

Model-by-Model Training Details

Each tab keeps the same structure: libraries first, architecture, training configuration, callbacks, metrics, and a compact code panel.

Healthy/Unhealthy

MobileNetV3 Lightweight Voter

F10.9206

Model RoleCompact binary voter that adds a lightweight architecture to the ensemble and reduces dependence on heavy backbones.

ArchitectureImageNet MobileNetV3 backbone, global pooling, dropout, sigmoid binary head.

ConfigMask-as-RGB input, threshold 0.50, healthy/unhealthy labels, Keras + ONNX artifacts.

CallbacksCheckpoint best validation model, early stopping, LR reduction, validation metric tracking.

F1	AUC	Precision	Recall	Notebook Peaks
0.9206	0.9730	0.9259	0.9153	val acc 0.9242, val precision 0.9381, val recall 0.9420

MobileNet architecture

base = MobileNetV3Large(include_top=False, weights="imagenet", input_shape=(256, 256, 3))
base.trainable = False
x = GlobalAveragePooling2D()(base.output)
x = Dropout(dropout)(x)
out = Dense(1, activation="sigmoid")(x)
model = Model(base.input, out)
model.compile(optimizer=Adam(lr), loss="binary_crossentropy", metrics=[AUC(), Precision(), Recall()])

MobileNet callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_loss", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=3),
]

MobileNet fine-tuning

history_head = model.fit(train_ds, validation_data=val_ds, callbacks=callbacks)
base.trainable = True
freeze_lower_layers(base, keep_top_blocks=True)
model.compile(optimizer=Adam(low_lr), loss="binary_crossentropy", metrics=metrics)
history_ft = model.fit(train_ds, validation_data=val_ds, callbacks=callbacks)

Healthy/Unhealthy

InceptionV3 Masked-Background Voter

F10.9160

Model RoleMasked-background binary voter that tests a deeper convolutional feature extractor with non-lung pixels suppressed.

ArchitectureImageNet InceptionV3 backbone with binary sigmoid head and fine-tuned upper layers.

ConfigMask-as-RGB input, background fill -1.0, threshold 0.50, Keras + ONNX runtime.

CallbacksValidation loss monitoring, checkpointing, early stopping, learning-rate control.

F1	AUC	Precision	Recall	Notebook Peaks
0.9160	0.9699	0.9244	0.9077	val acc 0.9171, val precision 0.9464, val recall 0.9562

Inception architecture

base = InceptionV3(include_top=False, weights="imagenet", input_shape=(256, 256, 3))
x = GlobalAveragePooling2D()(base.output)
x = Dropout(dropout)(x)
out = Dense(1, activation="sigmoid")(x)
model = Model(base.input, out)

Inception callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_loss", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=3),
]

Inception masked fine-tuning

mask_rgb = tf.repeat(mask[..., :1], repeats=3, axis=-1)
background = tf.ones_like(image) * -1.0
masked_image = tf.where(mask_rgb > 0.5, image, background)
model_input = tf.keras.applications.inception_v3.preprocess_input(masked_image)
prob = model(model_input)
label = int(prob >= 0.50)

Healthy/Unhealthy

EfficientNetV2B3 Four-Channel Voter

F10.9544

Model RoleStrongest healthy/unhealthy branch, using the lung mask as an explicit fourth input channel.

ArchitectureEfficientNetV2B3 transfer learning adapted for RGB plus mask-channel input.

ConfigNormalize to [-1, 1], threshold 0.85, best binary F1 and AUC among deployed binary models.

CallbacksBest-checkpoint saving, early stopping, LR scheduling, validation AUC/precision/recall tracking.

F1	AUC	Precision	Recall	Notebook Peaks
0.9544	0.9871	0.9641	0.9448	val acc 0.9574, val AUC 0.9873, val loss 0.2146

EfficientNet architecture

inputs = Input(shape=(256, 256, 4))
x = Conv2D(3, kernel_size=1, padding="same")(inputs)
base = EfficientNetV2B3(include_top=False, weights="imagenet")
x = base(x)
x = GlobalAveragePooling2D()(x)
out = Dense(1, activation="sigmoid")(x)

EfficientNet callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_auc", mode="max", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    ReduceLROnPlateau(monitor="val_loss", patience=3),
]

EfficientNet 4-channel fine-tuning

mask_1c = tf.cast(mask[..., :1] > 0.5, tf.float32)
four_channel = tf.concat([image, mask_1c], axis=-1)
four_channel = (four_channel * 2.0) - 1.0
prob = efficientnet_binary(four_channel)
label = int(prob >= 0.85)

Healthy/Unhealthy

DenseNet121 ROI Binary Voter

F10.9245

Model RoleROI-based conservative binary voter, optimized with two Optuna phases and a high decision threshold.

ArchitectureDenseNet121 backbone, ROI crop input, 1024/512 dense head from Optuna phase 1.

ConfigThreshold 0.91, label smoothing 0.075, dropout 0.25, unfreeze from conv5_block1_0_bn.

CallbacksTwo-stage training, warmup 3, checkpointing, early stopping, cosine LR fine-tuning.

F1	AUC	Precision	Recall	Optuna
0.9245	0.9711	0.9183	0.9308	P1 0.8882, P2 0.8968

DenseNet binary architecture

roi = crop_lung_roi(image, mask, target_size=(256, 256), threshold=0.5)
x = DenseNet121(include_top=False, weights="imagenet")(roi)
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation="relu")(x)
x = Dropout(0.25)(x)
x = Dense(512, activation="relu")(x)
out = Dense(1, activation="sigmoid")(x)

DenseNet binary callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_auc", mode="max", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    LearningRateScheduler(cosine_warmup_schedule),
]

DenseNet binary fine-tuning

loss = BinaryCrossentropy(label_smoothing=0.075)
optimizer = AdamW(learning_rate=8.824e-4, weight_decay=1.045e-5)
fit_classifier_head()
unfreeze_from("conv5_block1_0_bn")
set_lr(2.047e-5)
fit_finetune()

Disease Classifier

DenseNet121 Disease Sub-Classifier

F10.9782

Model RoleDedicated disease classifier used only after the ensemble predicts unhealthy, avoiding disease labels for healthy scans.

ArchitectureDenseNet121 ROI classifier with compact 256/64 dense head from Optuna phase 1.

Config3-class softmax: COVID, Viral Pneumonia, Lung Opacity. Dropout 0.45, label smoothing 0.025.

CallbacksCheckpoint best model, early stopping, cosine LR, class-weight power 0.163 for imbalance.

F1	AUC	Precision	Recall	Notebook Peaks
0.9782	0.9982	0.9781	0.9763	val AUC 0.9985, val loss 0.1672

Disease architecture

classes = ["COVID", "Viral Pneumonia", "Lung Opacity"]
roi = crop_lung_roi(image, mask, target_size=(256, 256))
x = DenseNet121(include_top=False, weights="imagenet")(roi)
x = GlobalAveragePooling2D()(x)
x = Dense(256, activation="relu")(x)
x = Dropout(0.45)(x)
x = Dense(64, activation="relu")(x)
out = Dense(3, activation="softmax")(x)

Disease callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_loss", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    LearningRateScheduler(cosine_decay_with_warmup),
]

Disease fine-tuning config

loss = CategoricalCrossentropy(label_smoothing=0.025)
optimizer = AdamW(learning_rate=2.798e-4, weight_decay=5.429e-4)
class_weights = compute_class_weights(power=0.163)
fit_classifier_head()
unfreeze_top_dense_blocks()
fit_finetune()

Architecture & Optimization

Architecture and Optimization by Model

Each model has its own architecture choices, input contract, thresholding, and optimization behavior.

MobileNetV3

Lightweight Mask-Aware Architecture

F10.9206

ArchitectureImageNet MobileNetV3 feature extractor, global pooling, dropout, and sigmoid head.

Input Contract3-channel mask-as-RGB input. Threshold remains 0.50 in deployment metadata.

OptimizationDesigned as a fast ensemble voter; simpler head keeps capacity controlled.

ArtifactKeras and ONNX outputs stored with MobileNet binary metadata.

architecture

base = MobileNetV3Large(include_top=False, weights="imagenet")
x = GlobalAveragePooling2D()(base.output)
x = Dropout(dropout)(x)
out = Dense(1, activation="sigmoid")(x)

optimization

loss = BinaryCrossentropy()
optimizer = Adam(learning_rate=lr)
metrics = [AUC(), Precision(), Recall()]
threshold = 0.50

InceptionV3

Masked-Background Inception Architecture

F10.9160

ArchitectureInceptionV3 backbone with global pooling, dropout, and binary sigmoid classifier.

Input ContractMask-as-RGB with background fill -1.0 before Inception preprocessing.

OptimizationSeparate Inception branch improves ensemble diversity through different receptive-field structure.

ArtifactKeras and ONNX exports preserve masked-background metadata.

architecture

base = InceptionV3(include_top=False, weights="imagenet")
x = GlobalAveragePooling2D()(base.output)
x = Dropout(dropout)(x)
out = Dense(1, activation="sigmoid")(x)

masked input

mask_rgb = tf.repeat(mask[..., :1], 3, axis=-1)
background = tf.ones_like(image) * -1.0
masked = tf.where(mask_rgb > 0.5, image, background)
model_input = inception_v3.preprocess_input(masked)

EfficientNetV2B3

Four-Channel Strong Binary Architecture

F10.9544

ArchitectureEfficientNetV2B3 adapted to consume RGB plus a mask channel.

Input Contract4-channel tensor normalized to [-1, 1]. Deployment threshold is 0.85.

OptimizationBest binary F1 and AUC; threshold raised for stronger precision/recall balance.

ArtifactKeras and ONNX exports carry 4-channel preprocessing metadata.

architecture

inputs = Input(shape=(256, 256, 4))
x = Conv2D(3, 1, padding="same")(inputs)
base = EfficientNetV2B3(include_top=False, weights="imagenet")
x = GlobalAveragePooling2D()(base(x))
out = Dense(1, activation="sigmoid")(x)

optimization

four_channel = tf.concat([rgb, mask[..., :1]], axis=-1)
four_channel = (four_channel * 2.0) - 1.0
prob = model(four_channel)
label = int(prob >= 0.85)

DenseNet Binary

ROI DenseNet with Optuna-Tuned Head

F10.9245

ArchitectureDenseNet121 on lung ROI crop with 1024/512 dense layers from Optuna phase 1.

Input ContractUses segmentation crop and DenseNet preprocessing. Deployment threshold is 0.91.

OptimizationPhase 2 tuned AdamW, LR stages, label smoothing, dropout, weight decay, and unfreeze layer.

Optuna ResultP1 best 0.8882, P2 best 0.8968.

architecture

roi = crop_lung_roi(image, mask)
x = DenseNet121(include_top=False, weights="imagenet")(roi)
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation="relu")(x)
x = Dropout(0.25)(x)
out = Dense(1, activation="sigmoid")(Dense(512, activation="relu")(x))

optimization

optimizer = AdamW(lr=8.824e-4, weight_decay=1.045e-5)
loss = BinaryCrossentropy(label_smoothing=0.075)
dropout = 0.25
unfreeze_from = "conv5_block1_0_bn"
fine_tune_lr = 2.047e-5

Disease DenseNet

ROI Disease Classifier Architecture

F10.9782

ArchitectureDenseNet121 on unhealthy ROI with compact 256/64 dense head from Optuna.

Input ContractRuns only after unhealthy ensemble vote; outputs COVID, Viral Pneumonia, or Lung Opacity.

OptimizationPhase 2 tuned AdamW, class-weight power, dropout 0.45, and label smoothing.

Optuna ResultP1 best 0.9241, P2 best 0.9583.

architecture

roi = crop_lung_roi(image, mask)
x = DenseNet121(include_top=False, weights="imagenet")(roi)
x = GlobalAveragePooling2D()(x)
x = Dense(256, activation="relu")(x)
x = Dropout(0.45)(x)
out = Dense(3, activation="softmax")(Dense(64, activation="relu")(x))

optimization

optimizer = AdamW(lr=2.798e-4, weight_decay=5.429e-4)
loss = CategoricalCrossentropy(label_smoothing=0.025)
dropout = 0.45
class_weight_power = 0.163

Callbacks & Fine-Tuning

Callbacks and Fine-Tuning by Model

Every model has its own staged training and callback contract instead of one generic callback explanation.

Stage 1

Classifier-head trainingFreeze MobileNetV3 and train the compact binary head on masked inputs.

Backbonefrozen

Inputmask-rgb

Headsigmoid

Stage 2

Light fine-tuningUnfreeze upper MobileNet blocks with a lower learning rate for final adaptation.

Threshold0.50

F10.9206

AUC0.9730

Control

CallbacksUse validation loss for checkpointing and early stopping, with LR reduction when validation stalls.

Monitorval_loss

Savebest

LRplateau

Persist

Deployment contractSave Keras, ONNX, metadata threshold, preprocessing flags, and binary class mapping.

FormatYAML

RuntimeONNX

Taskbinary

architecture

base = MobileNetV3Large(include_top=False, weights="imagenet")
x = GlobalAveragePooling2D()(base.output)
x = Dropout(dropout)(x)
out = Dense(1, activation="sigmoid")(x)
model = Model(base.input, out)

callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_loss", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=3),
]

fine-tuning

base.trainable = False
fit_classifier_head()
base.trainable = True
freeze_lower_layers(base)
compile_with_low_lr()
fit_finetune()

Stage 1

Classifier-head trainingFreeze InceptionV3 and train the binary head on mask-filled images.

Backbonefrozen

Fill-1.0

Headsigmoid

Stage 2

Top-block fine-tuningUnfreeze upper Inception blocks and continue with a lower learning rate.

Threshold0.50

F10.9160

AUC0.9699

Control

CallbacksCheckpoint the best model, restore best weights, and reduce LR on plateau.

Monitorval_loss

Savebest

LRplateau

Persist

Deployment contractStore background fill, mask-as-RGB flag, threshold, Keras artifact, and ONNX artifact.

FormatYAML

RuntimeONNX

Taskbinary

architecture

base = InceptionV3(include_top=False, weights="imagenet")
x = GlobalAveragePooling2D()(base.output)
x = Dropout(dropout)(x)
out = Dense(1, activation="sigmoid")(x)
model = Model(base.input, out)

callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_loss", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    ReduceLROnPlateau(monitor="val_loss", patience=3),
]

fine-tuning

train_head_on_masked_images()
unfreeze_top_inception_blocks()
compile_with_low_lr()
fit_until_validation_loss_stops_improving()
save_best_model()

Stage 1

Four-channel head trainingTrain the classifier with RGB plus lung mask channel while the pretrained body is protected.

Input4ch

Norm[-1,1]

Headsigmoid

Stage 2

High-F1 fine-tuningFine-tune upper EfficientNet layers and select a stricter threshold.

Threshold0.85

F10.9544

AUC0.9871

Control

CallbacksMonitor validation AUC for checkpointing and validation loss for early stopping.

Monitorval_auc

Savemax

LRplateau

Persist

Deployment contractSave the 4-channel preprocessing contract, threshold, metrics, Keras model, and ONNX model.

FormatYAML

RuntimeONNX

Taskbinary

architecture

inputs = Input(shape=(256, 256, 4))
x = Conv2D(3, 1, padding="same")(inputs)
base = EfficientNetV2B3(include_top=False, weights="imagenet")
x = GlobalAveragePooling2D()(base(x))
out = Dense(1, activation="sigmoid")(x)

callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_auc", mode="max", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    ReduceLROnPlateau(monitor="val_loss", patience=3),
]

fine-tuning

build_rgb_plus_mask_dataset()
train_classifier_head()
unfreeze_top_efficientnet_layers()
compile_with_lower_lr()
fit_and_select_threshold(0.85)

Stage 1

Optuna-selected head trainingTrain the 1024/512 DenseNet binary head on lung ROI crops.

Units1024/512

LR8.82e-4

Drop0.25

Stage 2

Partial unfreezingUnfreeze from conv5_block1_0_bn and continue at a very low learning rate.

Threshold0.91

F10.9245

LR22.05e-5

Control

CallbacksUse validation AUC checkpointing, early stopping, and cosine warmup schedule.

Monitorval_auc

Savemax

Warmup3

Persist

Deployment contractPersist ROI preprocessing, threshold 0.91, class mapping, Keras, ONNX, and Optuna JSON.

FormatYAML

OptunaP2

Taskbinary

architecture

roi = crop_lung_roi(image, mask)
x = DenseNet121(include_top=False, weights="imagenet")(roi)
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation="relu")(x)
x = Dropout(0.25)(x)
out = Dense(1, activation="sigmoid")(Dense(512, activation="relu")(x))

callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_auc", mode="max", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    LearningRateScheduler(cosine_warmup_schedule),
]

fine-tuning

fit_head(lr=8.824e-4, warmup=3)
unfreeze_from("conv5_block1_0_bn")
set_lr(2.047e-5)
fit_finetune(label_smoothing=0.075)
deploy_threshold = 0.91

Stage 1

Disease head trainingTrain the 256/64 disease head on unhealthy lung ROI crops only.

Units256/64

LossCCE

Classes3

Stage 2

Class-weighted fine-tuningApply class-weight power, AdamW, label smoothing, and high dropout from Optuna phase 2.

F10.9782

Drop0.45

Power0.163

Control

CallbacksCheckpoint best validation loss, restore best weights, and schedule cosine decay with warmup.

Monitorval_loss

Savebest

Schedulecosine

Persist

Deployment contractPersist disease mapping, ROI preprocessing, Keras, ONNX, metrics, and Optuna JSON.

FormatYAML

RuntimeONNX

Taskdisease

architecture

roi = crop_lung_roi(image, mask)
x = DenseNet121(include_top=False, weights="imagenet")(roi)
x = GlobalAveragePooling2D()(x)
x = Dense(256, activation="relu")(x)
x = Dropout(0.45)(x)
out = Dense(3, activation="softmax")(Dense(64, activation="relu")(x))

callbacks

callbacks = [
    ModelCheckpoint(best_path, monitor="val_loss", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    LearningRateScheduler(cosine_decay_with_warmup),
]

fine-tuning

class_weights = compute_class_weights(power=0.163)
fit_head(lr=2.798e-4)
unfreeze_top_dense_blocks()
fit_finetune(weight_decay=5.429e-4)
evaluate_macro_f1_precision_recall()

Model Training 02

InceptionV3 Healthy/Unhealthy Training

A binary model using Inception features and masked-background preprocessing.

Backbonetf.keras.applications.InceptionV3 with ImageNet transfer learning.

Input setupMask-as-RGB with background fill -1.0, so the model focuses on lung content.

Training signalsValidation precision and recall are high in notebook outputs, while deployment F1 is reported for model comparison.

Metric Group	Value	Meaning
Deployment F1	0.9160	F1 from precision 0.9244 and recall 0.9077.
Deployment AUC	0.9699	Useful ensemble member with different architecture and preprocessing.
Notebook peaks	val accuracy 0.9171, val precision 0.9464, val recall 0.9562, val loss 0.2833	Shows strong recall during notebook training, with deployment threshold retained at 0.50.
Artifact	`inception-binary.keras` + `inception-binary.onnx`	Saved with metadata for Keras/ONNX runtime selection.

Inception preprocessing

masked = image * mask_rgb
background = tf.ones_like(image) * -1.0
model_input = tf.where(mask_rgb > 0.5, masked, background)
model_input = tf.keras.applications.inception_v3.preprocess_input(model_input)

prob = inception_binary(model_input)
label = int(prob >= 0.50)

Inception training controls

setup:
  train classifier head first
  fine-tune upper Inception blocks carefully
  monitor validation loss and classification metrics

metadata:
  threshold: 0.50
  mask_as_rgb: true
  background_fill: -1.0

Model Training 03

EfficientNetV2B3 Healthy/Unhealthy Training

The strongest binary branch, trained with an explicit mask channel instead of only masked RGB.

BackboneEfficientNetV2B3, selected for strong accuracy/efficiency tradeoff.

Input setupFour channels: RGB image plus predicted or ground-truth lung mask as the fourth channel.

Comparison roleBest binary F1 and AUC, so it is the strongest individual healthy/unhealthy model.

Metric Group	Value	Meaning
Deployment F1	0.9544	Best binary F1 from precision 0.9641 and recall 0.9448.
Deployment AUC	0.9871	Best binary AUC among deployed healthy/unhealthy models.
Notebook peaks	val accuracy 0.9574, val AUC 0.9873, val precision 0.9799, val recall 0.9467, val loss 0.2146	Confirms the 4-channel strategy performed strongest before deployment export.
Artifact	`efficientnet-binary.keras` + `efficientnet-binary.onnx`	Largest binary model after Inception, but best F1/AUC.

EfficientNet 4-channel input

rgb = resize(image, (256, 256))
mask = resize(mask, (256, 256))
mask = tf.cast(mask > 0.5, tf.float32)
model_input = tf.concat([rgb, mask[..., :1]], axis=-1)
model_input = (model_input * 2.0) - 1.0

prob = efficientnet_binary(model_input)
label = int(prob >= 0.85)

EfficientNet controls

setup:
  custom input stem supports RGB + mask channel
  train classifier head before deeper fine-tuning
  threshold raised to 0.85 after validation review

comparison:
  primary metric: F1 0.9544
  supporting: AUC, precision, recall, accuracy

Model Training 04

DenseNet121 Healthy/Unhealthy Training

The ROI-based binary model and the main Optuna target for binary classifier-head and optimization search.

BackboneDenseNet121 with ROI crops from the segmentation mask.

OptunaBinary phase 1 selected 1024/512 dense units; phase 2 tuned LR stages, AdamW, dropout, label smoothing, and weight decay.

ThresholdUses threshold 0.91, making this branch more conservative than the 0.50 models.

Metric Group	Value	Meaning
Deployment F1	0.9245	F1 from precision 0.9183 and recall 0.9308.
Deployment AUC	0.9711	Strong ROI-based classifier with conservative thresholding.
Notebook peaks	val accuracy 0.9231, val AUC 0.9754, val precision 0.9786, val recall 0.9599, val loss 0.2945	Notebook training showed high validation precision/recall before deployment threshold selection.
Optuna phase 2	best value 0.8968, LR1 8.82e-4, LR2 2.05e-5, dropout 0.25	Selected final optimization behavior for DenseNet binary training.

DenseNet binary head

base = tf.keras.applications.DenseNet121(
    include_top=False,
    weights="imagenet",
    input_shape=(256, 256, 3),
)
x = tf.keras.layers.GlobalAveragePooling2D()(base.output)
x = tf.keras.layers.Dense(1024, activation="relu")(x)
x = tf.keras.layers.Dropout(0.25)(x)
x = tf.keras.layers.Dense(512, activation="relu")(x)
outputs = tf.keras.layers.Dense(1, activation="sigmoid")(x)

DenseNet binary staged fit

optimizer = tf.keras.optimizers.AdamW(
    learning_rate=CosineDecay(lr_stage1, decay_steps),
    weight_decay=1.045e-5,
)
loss = BinaryCrossentropy(label_smoothing=0.075)

fit_head()
unfreeze_from("conv5_block1_0_bn")
fit_finetune(learning_rate=2.047e-5)

Model Training 05

DenseNet121 Disease Training

A separate ROI-based multiclass classifier for COVID, Viral Pneumonia, and Lung Opacity after the binary ensemble detects pathology.

Task setupThree-class softmax classifier; Normal is excluded because healthy scans are handled by the binary ensemble.

Imbalance handlingOptuna phase 2 includes class-weight power, label smoothing, dropout, and AdamW weight decay.

Comparison metricF1 is the headline classification metric; AUC, precision, recall, and validation loss remain visible.

Metric Group	Value	Meaning
Deployment F1	0.9782	Best classification F1 in the page.
Deployment AUC	0.9982	Excellent class separation for the disease-only task.
Notebook peaks	val accuracy 0.9772, val AUC 0.9985, val precision 0.9781, val recall 0.9763, val loss 0.1672	Shows the dedicated disease classifier clearly outperformed the multi-task classifier for disease labels.
Optuna phase 2	best value 0.9583, dense units 256/64, dropout 0.45, class-weight power 0.163	Compact classifier head with stronger regularization.

Disease classifier setup

classes = ["COVID", "Viral Pneumonia", "Lung Opacity"]
base = tf.keras.applications.DenseNet121(include_top=False, weights="imagenet")
x = tf.keras.layers.GlobalAveragePooling2D()(base.output)
x = tf.keras.layers.Dense(256, activation="relu")(x)
x = tf.keras.layers.Dropout(0.45)(x)
x = tf.keras.layers.Dense(64, activation="relu")(x)
outputs = tf.keras.layers.Dense(3, activation="softmax")(x)

Disease training controls

loss = CategoricalCrossentropy(label_smoothing=0.025)
optimizer = AdamW(
    learning_rate=2.798e-4,
    weight_decay=5.429e-4,
)
class_weights = compute_class_weights(power=0.163)

train_on_unhealthy_roi_images()
evaluate_precision_recall_auc_f1()

Binary Classifier Family

Healthy vs Unhealthy Model Details

The four binary classifiers are intentionally different in architecture and input representation so ensemble disagreement carries meaning.

EfficientNetV2B3

F1 0.9544

Best binary performer by F1. It receives a 4-channel tensor where the predicted mask becomes an additional channel, allowing the network to see anatomy and the segmentation prior together.

DenseNet121

threshold 0.91

ROI-focused binary model. It uses the segmentation crop and a high threshold, making it a more conservative voter inside the ensemble.

InceptionV3

mask fill

Uses masked-background preprocessing with a background fill of -1.0, which suppresses non-lung pixels before the Inception feature extractor sees the image.

MobileNetV3

lightweight

Adds a smaller model family to the ensemble so the final vote is not only dominated by large backbones.

binary ensemble contract

binary_models = {
    "densenet": DenseNetBinary(threshold=0.91, use_roi=True),
    "efficientnet": EfficientNetBinary(threshold=0.85, concat_mask=True),
    "inception": InceptionBinary(threshold=0.50, mask_as_rgb=True),
    "mobilenet": MobileNetBinary(threshold=0.50, mask_as_rgb=True),
}

labels = [model.predict(image, mask).label for model in binary_models.values()]
final_label = int(sum(labels) > 2)
final_prob = mean(model.prob for model in binary_models.values())

model-specific preprocessing

DenseNet      -> crop_lung_roi + densenet.preprocess_input
EfficientNet  -> concat RGB image with binary mask channel
InceptionV3   -> mask_as_rgb + background_fill=-1.0
MobileNetV3   -> mask_as_rgb + identity float32 preprocessing

The API reads these choices from each model metadata.yaml file.

Disease Model

COVID, Viral Pneumonia, and Lung Opacity Classifier

Disease classification is separated from binary triage so healthy scans do not receive unnecessary disease labels.

Component	Choice	Reason
Backbone	`DenseNet121`	Strong feature reuse and good performance on ROI-focused medical images.
Classes	COVID, Viral Pneumonia, Lung Opacity	Normal is intentionally removed because the binary ensemble already handled healthy cases.
Input	Segmented lung ROI	Reduces shortcut learning from borders, labels, and background regions.
Output	Softmax probabilities	Returns disease probabilities and a label only when binary final label is unhealthy.
Deployment metrics	AUC 0.9982, F1 0.9782	The final page reports metadata-level results, not random notebook screenshots.

Class orderDeployment mapping stores 0 COVID, 1 Viral Pneumonia, and 2 Lung Opacity.

Trigger conditionThe classifier is skipped if ensemble final label is Healthy.

Response detailThe API returns disease label, disease probability, and full probability dictionary when disease inference runs.

Training Dependencies

Modeling Libraries & Dependencies

Each training family shares a TensorFlow base, with model-specific augmentation, masks, ROI logic, and Optuna search.

TensorFlow / KerasImplements transfer learning, distribution strategy, callbacks, custom metrics, progressive unfreezing, and saved .keras artifacts.

MatplotlibUsed for model training curves, validation behavior, segmentation examples, and notebook visual diagnostics.

OptunaSearches DenseNet classifier-head architecture and optimization parameters for binary and multi-class tasks.

Class weights + label smoothing + AdamWControl imbalance and overconfidence, especially for the disease classifier and DenseNet optimization phase.

CosineDecay schedulesUsed in optimized training phases to reduce learning rate smoothly after warmup and partial unfreezing.

Fine-Tuning Theory

Why Multiple Binary Models?

The ensemble is a deliberate risk-control design: different architectures see the same CXR through different preprocessing views.

Mask-Aware Diversity

ensemble

DenseNet and the disease classifier use lung ROI crops. EfficientNet receives a fourth mask channel. Inception and MobileNet use mask-as-RGB plus masked background handling. This creates architectural and preprocessing diversity.

Voting Rule

> 2 votes

final_label = 1[sum(model_labels) > 2]

All four binary models produce probabilities and thresholded labels. The final label becomes unhealthy only when more than two models vote unhealthy; the final probability is the average probability across models.

Two-Stage Diagnosis

triage

The disease classifier runs only after the binary ensemble returns unhealthy. This prevents disease predictions from being shown for scans that the triage layer considers normal.

Clinical Framing

assistive

The platform is a technical prototype for screening support. It exposes probabilities, per-model disagreement, segmentation visuals, and artifacts rather than hiding uncertainty behind one label.

Optuna Search

Architecture and Optimization Search

DenseNet binary and disease models were tuned with two Optuna phases: classifier-head architecture first, optimization behavior second.

Binary P1

DenseNet healthy/unhealthy architecture searchBest trial 12 selected two dense layers: 1024 and 512 units.

Best Value0.8882

LR3e-4

Dropout0.10

Binary P2

DenseNet healthy/unhealthy optimization searchBest trial 10 kept 1024/512 layers and tuned AdamW, label smoothing, dropout, cosine schedule, and staged learning rates.

Best Value0.8968

LR Stage 18.82e-4

LR Stage 22.05e-5

Disease P1

Disease DenseNet architecture searchBest trial 16 selected a compact two-layer head: 256 and 64 units.

Best Value0.9241

LossCCE

Dropout0.10

Disease P2

Disease DenseNet optimization searchBest trial 9 tuned AdamW, cosine decay, label smoothing, dropout, and a class-weight power for imbalanced disease labels.

Best Value0.9583

Dropout0.45

Weight Power0.163

Optuna search was used to select classifier-head architecture and optimization parameters, then persisted as JSON contracts.

Experiment Control

Callbacks, Fine-Tuning Stages, and Saved Contracts

The notebooks use controlled training stages rather than one uncontrolled fit call.

Stage 1

Classifier-head trainingFreeze the pretrained backbone and train the custom dense head so the new task-specific layers stabilize first.

Backbonefrozen

LRhigher

Goaladapt

Stage 2

Partial unfreezingUnfreeze the selected upper layers, lower the learning rate, and fine-tune without destroying pretrained visual features.

Warmup3

Unfreezetop

Schedulecosine

Control

Training callbacksEarly stopping, checkpointing, learning-rate control, validation metrics, and notebook plots make long experiments traceable.

Monitorval

Savebest

Plotscurves

Persist

Metadata as deployment contractEach final model folder stores Keras artifact, ONNX artifact, metrics, threshold, class mapping, preprocessing rules, and input shape.

FormatYAML

RuntimeKeras/ONNX

Sourcenotebook

callback pattern

callbacks = [
    ModelCheckpoint(best_model_path, monitor="val_loss", save_best_only=True),
    EarlyStopping(monitor="val_loss", patience=patience, restore_best_weights=True),
    ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=3),
]

history_stage1 = model.fit(train_ds, validation_data=val_ds, callbacks=callbacks)
unfreeze_top_layers(model, from_layer=selected_layer)
history_stage2 = model.fit(train_ds, validation_data=val_ds, callbacks=callbacks)

metadata contract

task: binary_classification
input_size: [256, 256]
preprocessing:
  use_roi: true
  concat_mask_channels: false
threshold: 0.91
metrics:
  auc: 0.9711
  accuracy: 0.9212
  f1: 0.9245

Saved Model Inventory

Artifacts Produced by Training

The research output is not just notebook metrics. It becomes a deployable artifact tree consumed by FastAPI, ONNX Runtime, MLflow, and Hugging Face.

Folder	Files	Runtime Meaning
`saved_models/healthy_unhealthy/densenet`	`densenet-binary.keras`, `densenet-binary.onnx`, `metadata.yaml`	ROI-based binary voter and metadata contract.
`saved_models/healthy_unhealthy/efficientnet`	Keras, ONNX, metadata	4-channel strongest binary voter.
`saved_models/healthy_unhealthy/inception`	Keras, ONNX, metadata	Masked-background Inception voter.
`saved_models/healthy_unhealthy/mobilenet`	Keras, ONNX, metadata	Lightweight MobileNet voter.
`saved_models/diseases/densenet`	`densenet-diseases.keras`, `densenet-diseases.onnx`, metadata	Three-class disease classifier.
`saved_models/segmentation/unet_xception`	`unet_xception-segmentation.keras`, `unet_xception-segmentation.onnx`, metadata	Lung segmentation and ROI creation.

Model Visuals

Architectures Used in the Research Stack

Selected notebook assets document the main transfer-learning backbones and segmentation design.

DenseNet

EfficientNetV2

MobileNetV3

InceptionV3

U-Net Xception

U-Net Skip Connections

Output Head	Saved Metric	Interpretation
`segmentation`	val Dice up to 0.9589, val IoU up to 0.9451	The shared model still learned strong lung masks.
`classification`	val classification accuracy up to 0.8887	Useful prototype, but weaker than the dedicated ensemble strategy.
`combined loss`	segmentation loss + classification loss	Demonstrates multi-objective learning, but final production path keeps models modular.

Artifact	Keras Size	ONNX Size	Runtime Use
`InceptionV3 binary`	257.3 MB	87.2 MB	Mask-as-RGB binary classifier.
`EfficientNetV2B3 binary`	157.2 MB	52.1 MB	4-channel RGB+mask classifier.
`DenseNet binary`	47.9 MB	27.7 MB	ROI-based conservative ensemble voter.
`Disease DenseNet`	47.8 MB	27.7 MB	COVID / Viral Pneumonia / Lung Opacity classifier.
`MobileNetV3 binary`	40.6 MB	13.4 MB	Lightweight masked binary voter.
`U-Net Xception`	23.8 MB	7.9 MB	Lung mask generator.

Model	Input Built From	Normalization	Reason
`segmentation/unet_xception`	Full RGB image	TensorFlow image scale	Predict a lung mask before downstream classifiers run.
`binary/densenet`	Lung ROI crop	DenseNet preprocessing	Focuses the classifier on lung fields only.
`binary/efficientnet`	RGB image plus mask channel	[-1, 1]	Combines visual content with explicit segmentation context.
`binary/inception`	Mask-as-RGB with background fill	Inception preprocessing	Suppresses non-lung pixels while preserving 3-channel input shape.
`binary/mobilenet`	Mask-as-RGB image	float32 identity / MobileNet style	Lightweight ensemble member with masked input.
`diseases/densenet`	Lung ROI crop	DenseNet preprocessing	Runs only after binary ensemble says unhealthy.

Route	Purpose	Important Behavior
`GET /health`	Health check	Returns service status and API version.
`GET /`	Root status	Returns service name, status, and version.
`POST /predict`	JSON inference	Accepts exactly one of image path, base64, or public URL.
`POST /predict/upload`	Multipart inference	Accepts uploaded image file and returns the same prediction schema.
`GET /logs`	Prediction history	Requires API key, supports limit and offset, returns DB-backed logs.
`/ui`	React frontend	Mounted as a static SPA with fallback to `index.html`.
`/static`	Artifacts	Serves prediction images and JSON mapping assets.

Mode	Compose / Runtime	Services	Purpose
`final`	`docker-compose.final.yml`	App, SQL Server, Kafka, Kafka UI, MLflow, Airflow, consumers	Complete local production-like topology.
`app`	`docker-compose.app.yml`	App, SQL Server, DB init, consumers	Application stack without separate MLflow/Airflow compose.
`runtime`	`docker-compose.runtime.yml`	Slim ONNX app + Postgres	Cloud-style deployment with Kafka and MLflow off.
`mlops`	`docker-compose.mlops.yml`	MLflow, Airflow, SQL Server, Airflow Postgres	Registry, backfill, evaluation, and scheduled retraining.
`kafka`	`docker-compose.kafka.yml`	Zookeeper, Kafka, Kafka UI	Event pipeline development and inspection.
`HF Spaces`	`.env.hf_spaces.template`	ONNX app, HF model download, Postgres, Supabase	Public deploy with external model/artifact services.

Table	Main Fields	Reason
`prediction_requests`	request_id, input_type, final_label, final_prob, final_probs_json, errors, created_at	Core prediction history.
`prediction_binary_model_results`	model_name, label, label_name, prob, probs_json	Per-model ensemble audit trail.
`prediction_disease_results`	label, label_name, probs_json	Stores disease classifier outputs when unhealthy.
`prediction_image_links`	source, mask, ROI, overlay URLs and paths	Connects logs to generated medical visualization artifacts.

Registered Name	Task	Promotion Metric	Artifact Source
`lung-binary-densenet`	Healthy/unhealthy	`val_f1`	DenseNet binary notebook + metadata.
`lung-binary-efficientnet`	Healthy/unhealthy	`val_f1`	EfficientNetV2B3 binary notebook + metadata.
`lung-binary-inception`	Healthy/unhealthy	`val_f1`	InceptionV3 binary notebook + metadata.
`lung-binary-mobilenet`	Healthy/unhealthy	`val_f1`	MobileNetV3 binary notebook + metadata.
`lung-diseases-densenet`	Disease classification	`val_f1`	Disease DenseNet notebook + Optuna JSON.
`lung-segmentation-unet-xception`	Lung segmentation	`dice_coefficient`	Segmentation notebook + custom Dice metric.

Env Template	Primary Use	Key Switches
`.env.template`	Local app defaults	SQL Server backend, Kafka, MLflow, Supabase toggles.
`.env.compose.template`	Full compose stack	Container hostnames, SQL Server credentials, Kafka brokers, MLflow URI.
`.env.runtime.template`	Slim ONNX runtime	`MODEL_RUNTIME=onnx`, Kafka off, MLflow off, Postgres DB, HF download on.
`.env.hf_spaces.template`	Hugging Face Space	`PORT=7860`, external Postgres, Supabase storage, CORS origins, HF model repo.

Test Area	Files	Purpose
API routes	`tests/api/test_routes.py`	Checks health, prediction route behavior, validation errors, and API response structure.
Unit logic	`tests/unit/`	Validates request parsing, preprocessing helpers, pipeline composition, and ensemble decision logic.
Inference service	`tests/services/test_inference_service.py`	Tests runtime inference behavior without depending on large external model downloads.
Model contracts	`tests/contracts/test_model_contracts.py`	Protects metadata, class mapping, model output labels, and deployment contract assumptions.
MLOps utilities	`tests/mlops/`	Checks ONNX release packaging and reviewed-data ingestion helpers used by operations workflows.

Pipeline Rule	Why It Matters	Project Impact
Tests before build	Stops broken runtime code before image creation.	Protects API and inference behavior.
Build before deploy	Confirms the runtime Dockerfile and dependencies still package correctly.	Protects Hugging Face Space release quality.
Secrets only in deploy	Keeps pull-request validation independent from production credentials.	Safer open collaboration and repeatable checks.
Health check after upload	Verifies the hosted service starts successfully after deployment.	Turns deployment into a measurable release gate.

Lung Disease Detection: End-to-End Medical AI Platform

Research to Production Architecture

COVID-19 Radiography Database Preparation

Data Engineering Libraries & Dependencies

Serialized Training Data

Dataset Checks Before Training

Modeling Libraries & Training Dependencies

Six-Model Research Program

Training Subsections and Notebook Coverage

Model-by-Model Training Details

MobileNetV3 Lightweight Voter

InceptionV3 Masked-Background Voter

EfficientNetV2B3 Four-Channel Voter

DenseNet121 ROI Binary Voter

DenseNet121 Disease Sub-Classifier

Architecture and Optimization by Model

Lightweight Mask-Aware Architecture

Masked-Background Inception Architecture

Four-Channel Strong Binary Architecture

ROI DenseNet with Optuna-Tuned Head

ROI Disease Classifier Architecture

Callbacks and Fine-Tuning by Model

InceptionV3 Healthy/Unhealthy Training

EfficientNetV2B3 Healthy/Unhealthy Training

DenseNet121 Healthy/Unhealthy Training

DenseNet121 Disease Training

Healthy vs Unhealthy Model Details

COVID, Viral Pneumonia, and Lung Opacity Classifier

Modeling Libraries & Dependencies

Why Multiple Binary Models?

Architecture and Optimization Search

Callbacks, Fine-Tuning Stages, and Saved Contracts

Artifacts Produced by Training

Architectures Used in the Research Stack

U-Net Xception Lung Masking

Libraries & Model Dependencies

Segmentation + Classification Experiment

Metadata-Driven Prediction Pipeline

Runtime Libraries & Artifacts

Export and Validation

Model Loading, Path Checks, and Warmup

How Each Model Receives the Same X-Ray Differently

API Startup, Prediction Routes, and UI Mount

Serving Libraries & Dependencies

Prediction and Response Contract

React Inference UI

Error Handling, Artifacts, and API Documentation

Docker, Kafka, MLflow, Airflow, Databases, and Cloud Runtime

MLOps & Deployment Libraries

Prediction Logging Schema

Prediction Event Consumers

Registry Backfill, Monthly Evaluation, and Retraining

Hugging Face, Postgres, and Supabase Path

MLflow Model Specs and Promotion Metrics

Full Stack vs Runtime Stack

Automation Libraries & Dependencies

Tests and CI/CD Pipeline

What the Automated Tests Validate

CI/CD Job Flow

Local and CI Execution Pattern

Project Synthesis & Roadmap