Get that information. And that is the overarching order that goes to the company. And complied with taskings. That asked my colleagues. The statutory term. And operational term to refer to Something Like an email or phone number, Legal Process, tasking being internal government terms for how you started the collection on a particular sector. Building on that what is the rationale for collections . If the target is the email account or the phone number, what is the justification for gathering communications between two persons who were discussing the phone number or email address, there is no to or from their email address or phone number. Why is that targeting permissible under the statute . The conclusion in a typical case, if you are targeting bad guy google. Com, you are targeting that persons account, their communications. It is not necessarily communication to or from that bad guy but that selector and so what the court has concluded is when the statute uses the term targeting a person overseas, targeting that selector qualifies under the statute for targeting nonu. S. Person overseas so it doesnt have to be targeting to or from but can target communication about that particular selector. The meeting of target earlier where you are focusing on an account and now you are discussing targeting discussion about that account to that is that selector. Is always focused on that account. The key is the misperceptions some may have that about collection, somehow about a keyword or persons that may be behind that account but all collection, section 702 whether it is upstream, a balance which is a subset of upstream or prism, based on the selectors of issue. My time is expired. Following your own rules. Just a followup on that, a good lighting query to make sure everyone understands, so you are saying if someone is emailing about rachel brand or explosives, that would not be a permissible about query under your explanation . I would like you could perhaps get it about rachel brand this is an issue all of us, clarify query in from collection, we are discussing now the collection of information, the collection of information, all collection of information is based, focused on selectors, not keywords like terrorist or a a generic name along those lines. And the same selectors used for the Prison Program are used for collection. Just a different way to effectuate the collection. Part of a large part of the function of these hearings is public education, function, i thought david ask questions were great, meaning different terms and i am glad you are willing to bear with us asking about questions we have already discussed in private but it is hopeful for everyone to understand why we are talking about and along those lines there was some discussion about purging data that doesnt turn out to be for an intelligence information. Can you explain how on the front end you implement the requirement that the target, nonu. S. Person reasonably believed to be abroad that you get for an intelligence information through the collection, a separate statutory requirement. How do you go about collecting that type of information . In our early discussion we switched to foreign determination that that is a seconds that. Has to be a reason one actually wants to collect intelligence in the first place and then have to get to this type of question permitted in the statute so it has to be about 4 intelligence to be in that collection but beyond that it has to be about 4 intelligence within one of the certifications approved annually. Certifications on things like counterterrorism, encountering wm ds, weapons of mass destruction. When an analyst needs to make a determination as to be valid for intelligence purpose for which they want to effectuate collection they have to document that. That is documented in a targeting rationale document, those are also reviewed by the Justice Department and director of National Intelligence every 60 days. An important point for non u. S. Persons, think about once you conclude a nonu. S. Person overseas and you collect everyone, that is not the case. Is targeted not only based on the identity of the person and the location of the person but that you are trying to get foreign intelligence sell it is an important protection in the statute designed for non u. S. Purchases. And not blanket collection for nonu. S. Persons overseas the only people on foreign intelligence targets. Growing up in that account, that will give you back information that is foreign intelligence like terrorism or proliferation or whatever it might be. What can you tell us in an unclassified setting about documentation of foreign intelligence or oversight . We talked a little bit about that. Can you give us anything more specific. They have to document that at nsa and essentially, tasking sheet and on that sheet they are documenting for intelligence purposes that they are trying to pursue by going after a party and a target and those are reviewed with foreign determination by the department of justice. For every selector. A review that rajesh de mentioned is under 30 days under these decisions you should say, foreign intelligence and foreign determination. The broader context, if the question is getting at the process within which that happened even before that happened we had training for analysts as to how they should document this material, we have audits in our databases, comprehensive compliance program, spot check within nsa prior to the 60 day review by the department of justice and quarterly reports about compliance to the program, semiannual reports to congress and annual Inspector General assessment, annual certification process. Those decisions are granular aspect of the program and conducted in the context of this broader regime. I see that my time just ran out. I wanted to ask one additional question about, can you do about collection through prism . No. Is limited to upstream. Prism is a collection to or from. To a separate topic, one of the things i have found concerning and frustrating through the process of our evaluation of programs is how do those articulate the advocacy of these programs, you had begun speaking about this in your prepared remarks. I would like to ask a couple of questions. One, how do you assess the advocacy of a particular program . How do you think we should be affecting the advocacy of a particular program . And it is not really a question but a comment which is please dont give me series of Success Stories and say that is how you advocate advocacy because that is an initial response from the government often in response to a question either from a body like ours or from the media. How do you assess the advocacy of the program . How periodically do you do so . How would you encourage us to assess the advocacy . Let me start on that and i will start by saying i completely agree with you that sort of individual Success Stories are not the way to evaluate Collection Program and utility. They way you evaluate Collection Programs is going to depend in part on what the particular program is for. Insists case we have, in fact, the director of National Intelligence has attempted part of our job is to try to determine resources are effectively allocated in the Intelligence Community budget so we have done studies to try to look at okay, what are our collection priorities, how much reporting is generated on these priorities and where do those reports come from . What collections force can we identify . And that is one of the ways we determine sections 702 is relevant. Another is by looking at the sheer nature of the information that we get and its utility to a whole variety of national priorities, that is a more impressionistic approach and yet you can see time and again, important intelligence reports provided to policymakers derived from section 702 so those are two ways that i would look at estimating the value of a particular collection. With respect to this program or any program i think intelligence professionals will tell you any tool must be evaluated in the context of the other tools for which it is utilized. All intelligence tools are used in a complementary fashion to isolate one particular tool and evaluate the second in isolation doesnt do us justice for what is valuable or what is not. It also depends on the type of tool. Different intelligence programs are used for different purposes. A program like section 702 is used for difficult purposes and the metadata program, one may be a discovery tool to provide more specific collection and others may be used as in fact the specific election that follows that. I think you really in order to understand whether its effective and useful you have to think about what your goals are with respect to this particular program and the goals for this program like many other programs are to obtain i think timely, accurate, informative, foreign intelligence information about the capabilities, plans, intentions of foreign powers, agents, actors and so on and so forth. Really what your talking about is i think developing a good metric to understand whether this program is worth all of the costs associated with it. So i think you would want to look at the amount of information that we acquire but also then the quality of it, how good it is. I think you can slice that a lot of different ways at my colleagues have suggested. I think thats what i would recommend you be focused on but because this is a broadbased foreign intelligence Collection Program you have to look at not only, you to look at counterterrorism but look more broadly than that because this program is not limited just to counterterrorism. I agree. I think the one point i should e behind having some clauses of there is programs is that when the statutory provisions expire, after the two the king program twice in the last two years as well as 702, congress undertakes anybody wishing of the effectiveness ofthe programs. I fully agree that it is an effective and Important Program and i really want to emphasize the last point you made, this program should not be considered still with the counterterrorist program. The program has significant and exceedingly important utility in areas outside of counterterrorism. Trying to clear up another issue. In terms of the participation of service providers, the awareness of service providers, the 702 implementation, is 702 implemented, all seven of the invitation is done with the full knowledge and assistance of any company from which information is attained, is that correct . Yes. The answer to that is yes. So early on in the debate, there were some statements by companies who may or may not have been involved in the program say we never heard of prism, but whether they ever heard of prism, any company that was from whom information was being obtained under 702 knew that he was being obtained. Spent correct the prison is just an interim government term that is result of the leaks became a public term but collection under this program was done consume a pursuant to Legal Process. And so they know that their data is being obtained . They would receive Legal Process in order to assess assist the government. One thing i read in one of the statements is under 702, you could target entire countries or regions are is that correct . So all collection under 702 is based on specific selectors, things like phone numbers or email addresses. It is not a bulk Collection Program. A selector would not be an entire area code . Correct. Going down to the constitutional oh, one other set of questions. Even ive lost track now of what youve already said youre versus what you said elsewhere, but in terms of where you make a determination that a person is, a nonu. S. Person outside, and then you later discover that that was good faith, wrong, the person was in the United States, or the person was a u. S. Person, and do you track that quacks and what do you do when you discover that, and how often do you discover im not saying about bashing im talking just about we thought he was outside the United States and that was just wrong. Or you thought he was a nonu. S. Person and that was just wrong. How often does that occur . Ill defer to brad on the sort of overarching review but if i could make a point about what happens. Yes, we keep track of every time the information comes to our attention to suggest that a prior intelligence evaluation was incorrect even if it had met the legal stupid every such instance is a matter of compliance matter that has to be reported to the disk and ultimately semiannual reports reported to the congress. And third disc. That such an progress encouraging progress by which information they should not have been collected it had not met ththe legal standards needed for some nsa systems. I think brad can speak to the level of accuracy. The item is the past and the information is perched. Suggested us to begin between two different types of blinds compliance issues. One is the rome example you mentioned to lets say were up on a cell phone that we believe belongs to a bad guy was outside United States, a foreign person. Then that person shows up in chicago. When that happens we ask the cell phone, that means we no longer collect communications at the complaint its that thats reported but its not an erroneous determination based on the movement of the individual. So putting those cases aside, in ca get it wrong, we think the email account with a focus located overseas but it turns out thats wrong or it turns out we think its a nonu. S. Person but it is a u. S. Person, we review every single one. Our review, as i mentioned, we think it is less than one in 1000 cases where they make that determination erroneously. It probably bears repeating, its not a one thing done so there is an obligation for analysts to reaffirm the foreign determination on a periodic basis, which contribute to the ability to make sure that determination is, in fact, fresh and current which, of course, introduce the accuracy of that determination. Going to the constitutional issues, the fisa court has determined, and they must determine every year that the program is being emulated consistent with the Fourth Amendment, the 31st time they determined that, there was an opinion the very first time they determined that, and i write, not yet public . I think thats correct. Isnt that a good candidate for declassification . We have a lot of good candidates for declassification. In all seriousness, we are there are a lot of documents that we have that we are reviewing for declassification that includes not only fisa Court Opinion but whole variety of other documents. Fisa court in 2008 when the less considered the constitutionality of the program, the predecessor to 702, the court issued a redacted but largely unclassified opinion conducting a relatively full Fourth Amendment analysis. And theres been some Fourth Amendment analysis conducted in this situation. And if youre sort of talking about, you know, the Rosetta Stone kind of document, then the very first Court Opinion should have been the most fulsome explanation of the constitutionality of the program. And i think, i mean i hear bob saying theres a lot of opinions out there, but to me this one seems to be one that would explicate at least one courts judgment on this because its been the basis of all, i assume all the rest of said nothing has changed that would merit us to reconsider our very first judgment . I think its among the opinions we are committed to reviewing all of the opinions on the fisa court to determine which ones can be declassified in redacted form to imagine this would be among those that are reviewed. So absolutely. It will be among the opinions that will be reviewed. I dont want to leave folks with any mysterious impression. I think the board has access to anything so should have to assume anything about subsequent opinions. The board in fact has reviewed everything. I just dont want what i can be an unfortunate consequence would be for folks to take weight impression that theres a mysterious opinion that its a secret analysis. And i dont think thats the case but i dont think you suggest that. The board has access but i think the question, the public should have access is part of the debate. Just one other thing to add on that. 702 collection is now been challenged by a number of criminal defendants. 702 information is being used against some of the cases so we will be filing public breeze and we can expect samore decisions in that area as well. Public briefs. Thats the way the cost and jolly of 702 will be on the Public Record, the opinions on it and the breeze and everything will now be in that record. Judge wald. My hope and under what substantive criteria is the initial decision to use a u. S. Person selector for searching the prism base, who decides lets do that, whats the substantive criteria on which they make it quick you dont have to go into the review process. I know the decision will be reviewed up and down. How does that get made . Whats the substantive basis . I can speak for nsa, particulates be just to clarify, that means if it goes to one of the other agencies, not nsa or cia or fbi, they make their own substantive decisions for query speak with yes. The 702 program is one that all agencies operate on their own and have their own minimization procedures which would address topics like searches. Nsa procedures in this regard, in his element have been made public. And so the standard in such a query needs to be briefly likely to return foreign intelligence information speak and who isnt made by initial . Made by the analyst. The analyst is working on that particular case, okay. My other question is, the president did, i understand did direct that there be some changes in the treatment of nonu. S. Persons asked to the limits on and retention of the data to bring them more in line with those of u. S. Persons, incidentally. Where there is no foreign intelligence value. Can you tell us a little more specifically, if anything has been done in that regard, or it is being contemplated, the city, 702 . So i think first of all its important to understand the point that somebody may have been bred made earlier, which is that there are already protections to