The hands of people we dont know. We crowd source on wikipedia. I rely on people on amazon to pie a book, and should i buy a play station 4 . Guide me to that. Im a nerd, but check this out. A spirted heated crowd source response how radioactive is our ocean . And they use crowd sourcing for the fukushima spillout. Raising money, totally different things. How likely are you to ask a stranger for help solving a serious personal problem . The internet has allowed us to do that, and with one click, we can tap into the collective knowledge of others to come up with solutions on a wide range of answers, and this is lead to breaks through in innovations, and critics say, proceed with caution, the information youre getting isnt always reliable. So how do you determine when more his are better than one . Helping to shape this question, dr. Alex lickerman, and welcome to the stream. Hi, lisa, and thank you for having me. Lets talk about human nature, what is it about us, as humans that makes us seek advice from others . So many things. Studies about all of the biases we have that drive us to ask other people for their advice and opinions. And so ill just mention two, off the top of my head, the propensity we have to ask crowds for their opinions and follow their advice. We have a task or a question in front of us that we dont know exactly how to answer. We have direct access to our own biases, weaknesses and downtowns, and we are in a sense ignorant of the answer and predisposed to not trust our own judgment. And we dont have access to the doubts and the foibles of people that we might ask. So they appear to be more knowledgeable than us. To have truth with a capital t. And so were automatically more predisposed because of our own weakens and biases to believe other people, and their judgments and answers over ours. And the second, i think, is that if you consider sort of the wisdom of the crowd, the crowd itself to be an authority in the sense, and just by numbers alone, we often think that the majority or crowd or any number of people more than one, us, has more collective wisdom. But also more collective authority. And there have been interesting studies showing that when we are brought in the presence of an authority and we submit ourselves to that authority, we submit to judgment of that authority. If we turn our judgment off, and we believe that the authority has Greater Knowledge and judgment than we do, we set it aside. So were predisposed psychologically to believe anything that authorities tell us. And speaking of crowds, we think were so cool use crowd sourcing but 150 years ago, the smithsonian launched a project where folks from around the country would submit weather reports into the smithsonian, and it was crowd sourcing in the 18 hundreds. And fast forward that now, you put the internet in the equation, and that changes the volume of opinions that were getting. Dr. Lickerman, are you fascinated by the obsession with not only our friends and neighbors, but the opinions of the world . Its amazing. We have never in our history had the opportunity to do this. To ask a question to not just hundreds of thousands, but millions of people who can weigh . In some way, based on the technology and the ability to express their opinion. Its fascinating. And mixed in with that has to be the biases that people bring, and the experiences, and you have any one of us, by using crowd sources technology, we have the expertise at our fingertips that we have never had access to. Its kind of exciting and thrilling, but something that we need to be aware of the limitations on. We have crowd sourcing in the digital age, lisa, and aria says i find crowd sourcing is finding those who agree with them and be happy that somebody thinks like them. And chris says so dr. Lickerman, do you think in the digital age, were increasing our opportunities by crowd sourcing . I dont know about increasing our insecurities, but a sense that we can label them more easily, and it may or may not be a legitimate sense. That if a large enough group of people agree with us, just the weight of that, it makes it feel like that opinion may or may not be true. And we need to be aware of the psychological bias that we bring to crowd sourcing so were not unduly influenced. But paying attention to the voice of the crowd when it is speaking the truth and giving us something valuable. Its a tricky thing. We know things like eyewitness testimony and an individuals ternal account of what happened can be shockingly inaccurate. I wonder if we have any metrics or data that show accuracy or inaccuracy of firsthand accounts of people online that is truly accurate or not . How many people believe everything that they read . Especially everything that they read online, and we know so much of it is not true. They can set up a site and verify their credentials and experience. What is the group youre asking . What is their background and expertise, and how confident are you that youre getting expertise . Theres a difference between putting out a tweet or Facebook Post that says the sprained ankle to your friends or exposing the public to a series of experts who are going to bring expertise. It depends on what specific crowd youre asking. The community is asking about it on facebook. The other day, i was trying to find the deal on traveling around europe with a u. S. Passport, so i did what any traveler would do we like that. What happens when the answers that you seek online involve your health . Solving the worlds most difficult medical cases, and what are the risks to finding what ails on you line. Being able to be a part and my dad contributing to research is an amazing experience. Its using information thats available, and then we provide to find a cure, and thats what we want. Welcome back, this is video from the website, patients like me. The healthrelated site allows people to compare symptoms and gather information. The founder of crowd med, it advertises medical cases, and phil bauman, a registered nurse who blogs about the roll of emerge the technologies and healthcare. So jarod, who is using crowd med . We have had 200 patients so far on our site. And most of our patients are people with chronic undiagnosed medical conditions, and on average, they have been sick eight years, and they have seen an average of three doctors, and incurred an average of 350,000 these are people with unsolved mysteries. Of these, how many have you solved . Well, its hard to track what we have definitively solved. But 80 of our patients tell us that they feel they got accurate diagnostic help from the sites. Doctor, are there averages on this . Its a cool idea. Working on unsolved mysteries, but there are, and im interested to hear. And one of them is to really when i come up against a diagnostic and i dont know what the patient has, i start over. I go back and sit down in front of the patient, and i retake the history. And i do a physical exam. And i know that a number of really tricky diagnoses will hinge on just one questionandanswer. And i know that the nuances that go on between a doctor and a patient will be lost on a site like this, and information will not be communicated. And as bright as these doctors may well be, they have to work from i be complete data, and theres no way to do a physical, which provides a crucial clue, and im interested to know how the site gets around that. Dr. Lickermans question is not an uncommon one. Every figures would rather put his hands on the patient and do their own history facing and their own physical examination, and obviously online thats not possible. But however, we try to make up with that disadvantage through various technical features. Information, anything to help them solve the case, and all of that back and forth is visible in realtime, o on the site. So our entire medical community has access to all of the data all the time. Jarod, you referred to it because you dont limit professionals, and are you concerned about not spreading this information . We made the decision early on that we wanted to cast as wide a net as possible. And the reason is insight that can lead the patient into their diagnoses and cure might come from anywhere. And not necessarily from the physician. About 65 of our active medical detectives do work in or study medicine, and however, we still welcome the 35 or 40 who have expertise on many diseases. Particularly in rare diseases, a lot of patients know more than the physicians do. We have a doctor tuning in, a former guest phil, youre a registered nurse, and i want to get you in this. Among other healthcare providers. The way that i look at crowd sourcing, we need to take a of browder, holistic indication of exactly what crowd sourcing is, and what are the conditions under which crowd sourcing may work or may not work. When you have different kinds of populations who are with different kinds of experiences, you could get different results. So if you think of crowd sourcing as a black box, and then you have input and output, and the difference, in the case of crowds, a patient comes in with a list of symptoms that go into the box, and you have healthcare providers, and then you get output. But depending on who you have in the box, and what kind of methodologies theyre following, and what other resources theyre using, and perhaps the sample size and the number, all of those things play a part in getting results. It has to go back to the patient, and then interpretation. And dr. Lickerman go ahead. Youre making a good point, but i wanted to bounce back to jarad for a moment, do you have any concerns regarding the people participate on your site . Could they be being paid by a Pharmaceutical Company or a Device Company and offering advice thats not as transparent as it should be . Were really careful about that. But theres always a risk that someone could go to our site with less than pure intentions, but one of the reasons we rely on the crowd opinions, we might get 50 medical opinions to weigh in an particular case, and the top diagnostic suggestions we provide to the patient are the result of a consensus. Among those several dozen people. And it would take quite a conspiracy to organize enough medical detectives to move the results in a selfserving way. But you dont have that many people participating all the time, right . So the medical detectives that i get today could be different than three days from now . Thats correct. So the set of medical detectives is unique to every case. We have a community of several thousand people, and several hundred are on the site at any given time. And we usually get 40 or 50 medical detectives in any given case. One of our detectives is a local detective dr. Licker man, i want to get you back in this conversation. One is the fact that physicians and nonclinicians are in on it. And it worries me. But one of the challenges that i have, they have already gone on web md, often, they have an idea and a notion that i have to spend a lot of time and its problematic that people get attached. The thing that concerns me the most, you have this crowd source of physicians coming to a consensus, and i assume the possibilities, or making transparent their thoughts, but one of the things differential diagnosis, for a couple of is young cancer, and they dont know this patient at all. And dont know that this patient has terrible anxiety. And they have included in the list the behavior, and they bury their head in the sand and they dont want to think about the possibility of cancer. So they dont follow up and get the appropriate treatment because from an authority they heard that they could have cancer. And they shut down, and theres no frankly doctorpatient relationship to buffer that reaction to that reaction, and that concerns me about how people are going to internalize the information from this site. Im curious again, if this has been thought about it and the steps being taken. Well get to the answer right after the break. Crowd sourcing does have 2s appeal for personal use, but what about professionally . One of the other things were going to talk about is the idea of corporations and government. Where does nasa come in . Working leading to innovation in government and business. Hi, im dr. Paul, the author of, and welcome to the stream. Were talking about strangers online to help solve problems, and risks and opinions from everything from what lawnmower to buy to what medical condition you might have, but are there applications for government . Nasaa one of the 50 federal agencies turning to the public for innovative ideas. Suggestions were again to them on how to make solar panels on the stas station more efficient. Urban dot us, you turn to the companies for the masses for problemsolving, and whats your perspective in terms of problemsolving . Most companies have their goto ternal r d, the people who tend to be the smartest in the room. And the question is, is there someone not there . And its not that nasa doesnt have the smartest people, but its interesting to see if theres anyone who could do better. So you could think of it as insurance in some cases, to make sure that you reach more people. Can you give us an example of innovations that have come to pass because of this approach. My favorite, the google robot car. That actually came out of dogman 2005. They ran a challenge, they said, if you have a robot car as one does, and you bring it to a desert outside of vegas and were going to have a race, and if your car competes in the trial and you win, if you solved a problem for them, good things happened after that. And so in the case of the challenge in 2005, no one actually competed the course, and then the following year, a team from stanford won, and turns out that team from stanford became the google robot car team. When you use crowd innovation, its not always the person coming up with the new idea but people feeding in it, or as raj said, me thinking in a direction that i wasnt going in, and its a Collaborative Experience . One of the things very often, intellectual property limits your ability to do that. So if you say, i have a certain amount available to award the winner, what if i do if raj had a really insightful thing that drove the condition. And its often not as much as you would like, but lego, going back to the segment with the video game reference, the video game called mine craft, was very popular in the beginning, this was a few years ago, and someone proposed lego on the site that they have that it should work with lego. And the way that lego checks that, we dont have to check everything, but well look at this if you can get 10,000 people to vote for it. So the mine Craft Community got together and said we like bricks in virtual space and in reality. Speaking of the future of crowd sourcing, we have carl. Carl from crowd reason. Com, you asked me about innovation in the crowds, and how to use crowd sourcing for innovation, but using a large set of answers from a larger crowd, you tend to get a broader answer set. And its really how do you filter through that and curate those answers, and thats what crowd sourcing innovation is all about. Casting those out and cure rating them to the right answer. Talking about curating, how do we make crowd sourcing more pratt and innovative moving forward . We have seen reputation systems, so the more in say a process for lego, the more you get points, the ideas that youve submitted. And how other people use those ideas. Amazon reviews, the more, the more useful reviews, and you have a list of the top 100 amazon reviewers, and that shows up in other domains, and its not that anyone gets to say that an idea is good. But its that the top maybe 5 get to say that. So dr. Lickerman, we have a minute left. I want to get back to this idea of personal crowd sourcing, and every time i go in the cafe, its next to impossible for the cashier to total my bill with his or her mind to give me the change back. But do you ever get concerns that our independence is going to soften our ability to critically think. Yeah, i do. Its easier for other people to do it for us, especially when theres so much knowledge that no one person could master it all. And its very easy to follow the temptation that theres some other person out there that we judge as better than ours, but theres a risk that we get so used to asking everyone in the crowd what we should think that were not thinking critically ourselves. Thanks to all of our guests font. Great show, well see you online. Good evening, everyone. Welcome to aljazeera america. Im John Seigenthaler in new york. Crisis in ukraine, hostages taken, and russian activists. Russia warns that civil war could be next. Conflict, International Sanctions are taking on russias economy. Equal pay. The white house with a defensive over a gender wage gap. Wrongful deaths. Hundreds of millions of dollars paid out for misdiagnosis and mistreatment of veterans