With UEFI you have both a Windows folder & an Ubuntu folder in the efi partition. the UEFI reads the efi entries and adds them to its own NVRAM to remember them. You have to remove ubuntu folder from efi partition first or UEFI will re-add it. Then you have to remove UEFI entry from UEFI.
You should have these folders in the efi partition. Delete only the ubuntu folder. Live installer should show folders. And if only Windows you have to mount from inside Windows the efi partition as it is not normally mounted.
You should not have to install Ubuntu but can use live installer DVD or flash drive. Some UEFI systems may let you do the UEFI edit from UEFI menu.
from liveDVD or flash and use efibootmgr
sudo efibootmgr -v
The “-v” option displays all the entries so you can confirm you’re deleting the right one, and then you use the combination of “-b ####” (to specify the entry) and “-B” (to delete it). Examples #5 is delete:
the full line will br sudo efibootmgr -b 0001 -B
//(here 0001 is ubuntu boot loader address)