Firefox downloads were failing, I couldn't delete files properly, Steam wouldn't update/install, there were random issues with moving files. That's just to name a FEW things that was driving me batshit insane over the past few hours with my new computer.
Get out of my way! I ain't in the mood for your shit!
An easy way to test this is to copy a new exe file somewhere and try to delete it. It may disappear, but when you refresh the folder it'll still be there. Checking out the locks on Process Explorer will show 2 processes locking it, System (pid 4) and Explorer.
For some reason, this manic behaviour was all caused by disabling the "Application Experience" service. Simply re-enable it and everything should just work again.