The fact is that this scenario never should have been presented
You say that, but nations often have plans of what to do in various unthinkable scenarios. As an example, the US put together various war plans in the 20s and 30s for fighting various nations - including the UK. Given the US and UK were on friendly terms it seems unconscionable to put together a plan on how to fight, but it's far better to do it while you're friendly and have time to think than after the shooting starts.
Arguably being in a shooting war is a more serious than just losing a technological edge[citation required], but then economic espionage is also less serious than shooting. However I can see that any dropping behind is seen as the start of a slippery slope to a loss of world power so early counter-measures are needed.
To me it's less that this scenario shouldn't have been presented, and more another example of US hypocrisy being revealed
But isn't arguing that the goal of an action can justify the action the same as claiming the end justifies the means?
To quote Wikipedia: In an extreme form, the idea of consequentialism is commonly encapsulated in the English saying, "the ends justify the means", meaning that if a goal is morally important enough, any method of achieving it is acceptable
From your comment, with reference to US and ISIS waterboarding, you state that "The difference is in the purpose" and "those in the employ of the CIA and the drone strikes are not and did not commit brutality with the primary goal of dehumanizing and enemy and publicly attempting to strike fear in the them with their brutality"
To me this reads that there is a difference in the evilness of waterboarding depending on the purpose - and therefore that the ends can justify the means?
There are different degrees of wrong and evil, but the difference lies in the action not the objective.
The end doesn't justify the means, so whether you are doing it under the belief that it will help protect your nation, or to strike terror in your enemies doesn't matter. If the action is the same it is just as good or evil no matter why you are doing it.
To quote Christopher Dawson: As soon as men decide that all means are permitted to fight an evil, then their good becomes indistinguishable from the evil that they set out to destroy
Note that I'm not saying that the US is as evil as ISIS - ISIS has done many things more evil than waterboarding. However the US cannot claim that their use of torture was acceptable because they did it for noble reasons. After all I'm sure ISIS believe they are torturing people for a noble reason.
The Data Retention Directive 2006/24 (DRD) said that member states have to store everyone's data
However Article 7 of the Charter of Fundamental Rights of the EU says that "Everyone has the right to respect for his or her private and family life, home and communications".
In June the DRD was found to be invalid due to the conflict with Article 7, so member states are busy constructing ways to still do mass surveillance without breaching Article 7.
These latest documents however suggest that (one of) the key points why the DRD breached Article 7 was it didn't require a link between the person being surveilled and a threat to security, or a particular place/time.
Therefore in order to avoid breaching Article 7, any new data retention measures have to be targeted in some way.
Therefore, no blanket surveillance of everyone.
I'm sure our "security" services are busy working out loopholes. GCHQ could define "the world" as a specified geographic area but that would let those terrorists on the ISS go uncaught - and they sound like they are linked to the ISIS guys in Iraq. Instead GCHQ could take a leaf from copyright and define a limited time to be forever less a day.
I'm not a lawyer so I don't really understand that para 59 quote above. I had a look at the link but the longer document is just as confusing.
To me, para 59 says that the data retention directive doesn't require a link between the data and threat to public security, and then gives a couple of examples in particular.
Why does that suggest blanket collection is prohibited? Saying a link isn't required doesn't seem to close off anything. Saying a link must or must not be present would start to close off options, but just saying a link could be there but doesn't need to be isn't helpful?
Someone could set up a search engine to do the US and EU searches for you. I think there was a version of google image search someone had done which would run the same search twice - once with safe filter on, and once with it off - and then just return the pictures which were filtered.
Instead of a porn-mode image search, how about a Streisand-mode web search which just returns the stuff people want deleted?