A Nuke is a type of antiquated denial-of-service (DoS) attack carried out by sending fragmented or corrupted (usually ICMP) packets to a target machine. For any machine running an older more vulnerable operating system, sending such packets to it will slow down and eventually stop it, resulting in a crash or Blue Screen of Death (BSoD) in the case of Windows.
Perhaps the most famous example of a Nuke attack was the 1997 WinNuke, which exploited vulnerability in Windows 95 that allowed for an attacker to cause a target machine to BSoD by sending it a string of invalid data on TCP port 139. The vulnerability used by WinNuke was patched by Microsoft a few weeks after its source code was released. Years later in 2002, a newer version of WinNuke surfaced that affected Windows NT, 2000, and XP, but it was quickly patched by Microsoft. Almost no modern operating systems are vulnerable to such an attack.