Federated Learning Project Reveals High Accuracy Can Mislead Detection
A federated learning comparison shows that high global accuracy can mask poor detection performance.
In a research project, federated learning algorithms (FedAvg, FedProx, FedNova) were compared against a centralized baseline for network intrusion detection using the CICIDS2017 dataset split into four silos by attack type. One silo, Web Attacks, had only 3,000 samples out of 3 million total, creating a significant imbalance. While global accuracy appeared high, it masked poor performance in the minority silo, which had a mere 49% accuracy and missed all attacks. FedNova showed greater consistency across silos, highlighting that global accuracy alone is not a reliable metric in federated intrusion detection; per-client performance and aggregation methods are crucial for detecting rare attacks.