Transcripts For CSPAN2 Andrew Blum The Weather Machine 20240714

Technology used to develop daily weather reports. From politics and prose bookstore in washington dc this runs one hour. [inaudible conversations] good evening everyone of the coowner of politics and prose welcome thank you for coming and congratulations on getting here what a great day to have a book about the weather. If yousn have been following the weather you will have noticed at least two things, first is getting weirder and wilder with bigger storms and freezing temperatureses and greater extremes and second, forecasters are Getting Better at predicting what is about to hit us they still dont get it right 100 percent of the time so with the timeliness of their reports. These days not only of one weather model but of multiple ones simulating the atmosphere the north americanri model the european model, and a number of others forecasting not just todays weather or tomorrow that over the next ten days and beyond. Getting interestedlun he wasnt satisfied with the past explanation to attribute the advances of weather science with the supercomputers or better satellites with a knack for making complex systems accessible which was a primer on the physical infrastructure on thet internet and the Weather Forecast was a similar kind of story. His new book the weather machine is an engaging look at behind the scenes of the International Collaborative effort and it takes to give us the daily weather report. Forecasting has been involved to the Meteorological Organization to introduce readers to the supercoerresting. He will be in conversation this evening with one of washingtons own expert forecasters the weather editor of o the Washington Post and chief meteorologist of the capital weather gang in jason himself is fascinated since his days growing up in the dc region as a High School Student he interned with bob ryan that will recall was the chief meteorologist for the local nbc affiliate and went on to study Atmospheric Science then worked as a Climate Change analyst with the epa. Fifteen years ago established capital weather. Com the first professional weather bloglo on the internet the Washington Post took the blog in 2008 now essentially for anyone who is interested in the weather in the dc area. Please join me to welcome both andrew and adjacent. [applause] hello. Thank you for coming in that great introduction. Im glad to have the chance with the new book on the train this morning jason definitely had a busy day. He agreed to be the interlocutor tonight we first met several years ago when i went to the meeting of the American Meteorological Society in atlanta and knew i wanted to write about this topic trying to figure out the shape and having all the fmeteorologist in one place was definitely a great starting point. What i didnt realize is that there was weather Journalism Community the second i was there i was warmly embraced to go out to about 25 people like ethe American Weather press corps. I think the dean. [laughter] but it was a big help starting off. And then we will open for questions. Its a pleasure to be here tonight the first time i ever took in huber from the ams in that meeting in atlanta to the airport back in 2014. Five years ago. As preparation for this event i read the book but it was a joy. I found myself smiling like sometimes you watch a great concert or singer and a smile comes across your face because you are amused how powerful the performance is. The writing is elegant with a lot of excellent narratives and anecdotes. It does get my high recommendation. So lets get started with q a. So lets begin with motivation for the book and what makes you decide all of a sudden you wanted to write a book about Weather Forecasting . My background is writing about architecture and places and you cant write about places without thinking about the weather but thats very hard to write about especially if you are not a meteorologist or atmospheric m scientist. The words slip away with a constant challenge between the view get your umbrella and the catastrophic and that combination to meteorologist to writean about the weather in deal in any daily basis to switch registers from the cherry to the catastrophic is the biggest challenge so knowing that i wanted to write about the weather was that Hurricane Sandy in 2012 which was the moment for all a lot of us the weather models suddenlythemselves there was a public conversation about the european and American Weather model and astonishing forecast it is sunday afternoon was the first inkling for sandy which arrived theh following monday night and that exceeded all my understanding of what meteorologist were capable of not only theirni human ability but their pattern recognition and the manual assimilation earlier they built systems that were astonishing that far exceeded my understanding of how they worked but yet the question of the weather models and how they work was very hard to answer. This incredible black box and that type of complex infrastructure story to recognize if i could figure out how these models work or to tell a story about them that is meaningful and a story to be told and i may have underestimated their complexity. Everyone one anybody here work with weather models . There is a very small pool of people who work on them and improve them. The entry to recognize their part is significant but to check the boxes so important for me but just the system we touch every single day with the idea that there is incredibly complex computer we ckcarry in our pocket and here its a story i wanted to tell that you cannot tell without going back in time with the history and the way that i approach that. Lets talk about the history and those pioneering scientists in the 18 hundreds that have modern day numerical weather prediction though scientist in england and norway that those early contributors and how important the work they did was to the advancements of weather predictions today. There is the history of the storm a chasers and these heroic stories of wind speeds and seather maps and then another history of mathematicians and when i start to think about how to tell this other history it isrtel distinct and it very much begins and its amazing because he occupies both and comes up with the idea in the turn of the 19th century to calculate the weather and use those equations to describe the atmosphere and what it would do next with the insight that each days calculation if you could calculate the weather you can prove the next day if you were right or wrong. And then do that on a daily basis which is the process of improvement that has got to us to the incredible point we are at today. But to say so those models ashave improved by a decade or over the last 40 years so it was the four day forecast a few years ago but that rate is not diminishing it is increasing with a two week forecast by 2025. So this improvement goes back to the idea each days forecast is a science experiment and you can try again the next a day. But the mathematicians actually tried and realize you would need 64000 computers which is 64000 people which could be arranged in a stadium and they each get their square of atmosphere and that is where they would do the calculations and pass them up to the front and maybe that would work to calculate the weather fast enough of course it has to be donee before it comes or is not useful. So that in the ability to calculate or those lack of computers to calculate essentially meant their ideas had to wait 50 years until computers and satellites came bin and then another 50 years for the models to be useful beyond the human scale not just guidance but to exceed the capability of meteorologist so thehe now yes there is a human check but many will still say they dont work. It is remarkable how goodtr those forecasting models are today forecasting down to the second so one of the developments as we fast forward to the fifties and sixties talking about weathert satellites so talk about the history and how they played such an integral role and the major discoveries and breakthroughs with that technology. The Key Technology for longerrange forecast, you need a global view. You really talk about the weather globally. So for a global view you need a global instrument. Some in the sixties you need to have the first inkling of that. So it is the month of apollo but they come out at the same oconjoined military effort without the dollars spent for the missile race no weather tsatellites and the scientific ideasli that are hand in hand. Until quite notably with kennedy the moons shot speech that we put a man on the moon before the decade iss out was bullet pointr number one. Number two was Nuclear Rockets for deep space travel. Have this basic impetus for a global view in the same speech is the moonshot speech and the most famous interceptor speech and then you have it based rooted in the idea of the International Cooperation that kennedy like the weather because it was a point of cooperation and sure enough, the thing is that speech in the later speech in the fall at the un that this annihilation speech was answer that can be proposed for togetherness for an alternative for peace was cooperation on weather observation and whether operation in control got left behind. That idea that from the very beginning it was about diplomatic cooperation between meteorologists from every country in order to make global data out of Global Infrastructure to eventually support global models and gives this route of cooperation that begins the 19th century but current incarnation was very much inspired by kennedys idea of a global vision. That is pretty or continues today and someone quietly and i think we often dont look beyond American Weather service but everything depends on this global pool of data. Absolutely. Clearly the data. So critical to todays weather models whether you talking about the weather balloon data or groundbased sensors and of course you just talked about Weather Satellites and Weather Satellites are superexpensive and cost of government billions of dollars but without them our forecasts would not be where they are. Obviously the Weather Satellite data observations from groundbased sensors, weather balloons, they beat these models and you went to two Different Centers to learn more about weather models but went to the National Center in boulder and the European Center for forecasting i did not put that in the book. Talk about your experiences and the appreciation game for these models and touched this out a little bit. And what makes these models work and what amount of intellect is required to run these models and who are the people who are doing this work and how have they been able to be so successful in improving Weather Forecaster for time . This competition between american and european model is short but whats amazing is visiting as a journalist the cleaner im looking for cleaner stories for more legible stories and its amazing to me the cleaner stories come from cleaner places. The places that have a coherent organization and has a very focused Mission Statement are much easier to describe and write about. They are often, i find, more successful because of that. With weather models the fact that we take the starting point that okay theres this weather model and its the best to see or go visit the source of that European Center which is outside of london in redding and the collaboration funded by i think its 32 yes, 32 European Countries that are contributing sending both scientists in money with this singular goal of running the best Global Weather model. They do it by combining the Research People and whether model operation people are single buildings and its amazing cafeteria has become a cliche in the tech culture but i cannot believe anyone got work done because the cafeteria was always full. They had the most beautiful coffee machines. [laughter] that constant day by day effort to consider what the model was doing and what he was spitting out and how it could be improved as a system was really tangible where it is built in not just in daily but internal wiki of comments on this is weird so why is it doing this and to codify with weekly meetings one of which i observed and wrote about and this constant sense of what is a model doing and how could he be doing it better. The key point is that when you say or talk about it is a well mother whether model of convocation meatgrinder where the present goes in and future comes out but its about an ongoing simulation of the atmosphere that every six hours or 12 hours is compared to the most recent observations the real atmosphere and correct it slightly to better match that and that duet of the simulated atmosphere clicking Forward Together with the simulated running faster time to give us what we call forecast. It makes you realize to make the forecast better means to make the simulation better. Not just in some statistical improvement based on past weather but more about actually using the equations to simulate the atmosphere in a way that is close enough to reality that it can then when its run forward forward faster than time it spits out the weather of the future. To see how eager the people at the European Center were to make sure they were getting it right which is not just to say they were getting the forecast right but to say that the simulation they were building most closely resembles the real atmosphere for all the right reasons. The actual physics maps not just getting lucky repeatedly and thats not Machine Learning and not a project problem in the way elections or other sort of baseball or other protected problems but its unique and uniquely successful. This is something we all use everything that they mostly right. Does that answer your culture question . , to divide they were competitive and want to be to the other modeling centers around the world . Thats all they talked about. [laughter] no, when you say why are you doing this they say no and that was theres a lot of ongoing discussion about the modeling competition and there is a sense that it keeps everyone that it lights the fire and keeps everyone improving their models faster but there is a sense that they need or do it and they the u. S. Models are convoluted by having to serve different masters in different models in everyone says we run 12 models and they only run one but while that is true the singular focus on this basically tenday simulation of the atmosphere makes their collaboration and their competitiveness very productive. And much more of a streamlined process. So, obviously we talked about how great Weather Forecasts are but theres a saying here that your Weather Forecast can be one 100 accurate but if people make the wrong decision based on an accurate forecast the forecast is useless. That was when i began to understand that about what makes a good forecast. How does the forecast informed decisionmaking . Does the forecaster provide the end user the information they need so that if theres a wedding reception coming up that they will go to for example, they will have a tent or not to have a tent to keep their guests dry. You talk about that decisional support expect in your book and you think Weather Forecasting centers both in the european and the United States gain a better appreciation of the importance of the social lines connecting the physical science that is the forecast with the decision . Is the place where the american not Just National Weather Service but broader enterprises is excelling is this emphasize these new links between this forecast and the tactical sense of the decisions you make based on it. I think i write about a guy named tim palmer who was at oxford in at the European Center but he tells this amazing story about its not as simple as will it rain or not lorraine but how sure in what decisions can be based on that . The story he tells if youre having a picnic and the queen is coming 20 is just enough to order ten but if its 80 for just friends then you might not so its that sense of how or what the use of a better forecast if it doesnt help our decisions. That is reasonable with the daytoday forecast just because it was clear it would rain monday morning in dc from four days ago but was not clear how it would rain and we were talking before at what point this morning does it seem like things are getting intense and unfortunately what decisions with people have made differently had this days forecast and clear. That idea that you can strive technically for the perfect forecast but if it does not help you make decisions meteorologists love it because it acknowledges that they are right and the ball is in some ways out of record but its only out of the court if you recognize that the technical superiority and technical achievement of the forecast itself. But you still get angry emails. Yeah, for sure. I think Weather Forecasts have come a long way and were good at capturing what we call synoptic scale weather systems, largescale systems from lowpressure systems but when it comes to forecasting individual thunderstorms this is what we had this morning models are still just developing in that area. We have what we call convection allowing models which try to get thunderstorms and small scale weather features correct. InterNational Weather community where we will be collecting data in the future. There are more resources coming online and companies that want to so their data. You have the remote census systems on automobiles. So, talk of little bit about this issue coming up in the organization as to historically whether the data has been a Partnership Among government but now you for introducing the private sector which as a privatesector motive into this enterprise, how do you deal with that, and talk about how your book addresses the. Its something i realized the folks involved in the daytoday of the forecasting in the u. S. In particular. Because the International System worked so well and evaporated quietly, that is what a woman at the Weather Service sort of sad. Our collaboration of exchanging weather observation over the world to send them back over the world has worked so well that we kind of moved the radar and people dont appreciate how essential that is, and i think that is the peace the fact that it isnt recognized i think becomes a cause for concern when you have these new kind of weather observations. So, when in particular those that are no longer after 150 years of meteorology being something done mostly by governments and weather observation and weather stations operate mostly satellite, mostly by government, you have a shift towards the private company observations and the tradition has always been to share their data and get lots of data back. The more recent evolution is a privatprivate satellite companis eventually offering data subscription and the question then if they sell to one government can that government share it with every other pitch is a challenging Business Model if you are running a Satellite Company because obviously you want to know a data to as many people as possible. On the flip side if you cant share th that data with other governments, then you have a risk of fracturing the entire system where the exchange is shut off and you end up with fewer observation that the best of improvement because you have all of these new observation streams that are no longer a part of this pool of Data Incorporated into the global model offering the forecast. That is a challenging place to be. It is a shift with the bigger bucks involved and more money at risk were more money to be made with weather streams and so the night before it was sort of unheardof that a company with one flights on thwouldfund fligr satellite or spend 20 million on the supercomputer models. Now those dollar values start to seem more reasonable because of this combination of the climate risk and more capabilities of the better forecast itself creates a business opportunity, and that there have been private forecasters for a while. Others have been around for a guest 20 or 30 years, the shift would be no longer about the global pool of data for the Companies Taking the data for better forecast, but instead about a sort of fragmentation of the new kind of data that hasnt been shared. Its uncharted territory and something that the folks who are the representatives are thinking hard about. And its something of course that the Current Administration had been geared to push for a in the interview recently that is the emphasis and it isnt a unique emphasis, its not a unique desire to dictate unique emphasis. But sort of balance has shifted to say we want to make sure these new companies can blossom, and its not clear what the stakes are for that for the Data Exchange since the tradition of the data to the public good. So, its coming and accommodation of the factors i think the macro factors have to grow very clear and its under the radar in the weather observation. Its interesting because there could be a day for example there would be a better forecast and the National Weather servi service. They are developing their own model and they have brilliant scientists who can do their own supercomputing and sell that information, so the Weather Forecasting which historically has been a public good. If you want the information rather than getting it for free, you maybe have to pay for it and theres a potential that the publicly provided Weather Forecast could become inferior unless the Weather Service develops partnerships with these Partner Companies in a way that they can merge their agreements. The reality is if it were not for the predominantly when you think about the new satellite data, if really the European Consortium thats the American Weather satellites that contributed the lions share of data, not just the pictures that were used from the satellites, but a lot of the numerical data that comes from the other satellites, the polar orbiting satellites. When you think of how big the contribution is the billions of dollars if list goes for the global meteorology. And if that exchange stops, then you would have at most ten or a dozen countries that would be willing to pay for the data and that would be that. I can see the factors coming together to prevent it and it is strange to think about because it would be a 150yearold tradition. Because of a privatesector organization that had the best forecast and they knew a lifethreatening storm is going to get before the national Weather Service, would they be ethically obligated to provide the forecast to everyone free of charge to the service that has a mission to know about it and getting into them so they could disseminate it. So if does raise questions like that. Talking to some of the people around the world are most concerned about these changes, how deeply ingrained sense of meteorology in the public good is and to see the sort of shift from public good to commodity. Should we take some audience questions . If you have a question could just raise your hand and i will pass the microphone over to you. Fascinating talk. In regards to why we should care about having longer and more accurate forecasts i was wondering if you could do that with regards to disaster preparation and how much did the government know it was going to come two weeks in advance as opposed to one week in advance . I think you see these moments where its not just a storm wasnt forecast for days before or three days before the time to make a decision based on it, but thats because the forecast was so consistent for so long for the confidence and making a decision on this was greater. The main example is from early may there were similar storms. This time around not just the longrange forecast, but the forecast event was several days out. The confidence in the forecast to be ablfor people to evacuaten people was clear. The idea that you have a high impact with a twoweek warning come if that forecast is reliable and we have a weeklong warning, the counterexample this spring was in mozambique where the storm had catastrophic damage and was predicted. It wasnt predicted with enough time or confidence for the local government to act on it. I also think that the dissemination system in the way of getting the word to the public and the Warning Systems were not as sophisticated as they are in india because they have a lot of experience having dealt with them repeatedly over the years and they learned from experience they have had absolute devastation, hundreds of thousands of people died, so they put into place Warning Systems and plans to get people away from the coast. Its interesting i just want to make one point. For the hurricanes obviously having a lovely time it is incredibly valuable in the forecast theyve done so much better compared to where they were in the 1970s and 1980s, which 24 to 48 hours with the good specificity now compared to where the word, but whats interesting is with tornadoes, theres been research that has shown you give people too much lead time and actually make worse decisions, so maybe the sweet spot might be 15 to 25 minutes, somewhere in there. The people know i know if you go something right now because in two hours they can make bad decisions and decide they want to wait until the last minute and drive away. So there is interesting social Science Research and again its going to this bridging the gap between the forecast in peoples decisions. The example this spring with the tornadoes in ohio and missouri where there was a major tornadoes and read the various is a sort of real victory for the national Weather Service going back to the joplin tornado where 160 was killed and again in the moments where it isnt just about the duration of the forecast but the fact that if it comes confidence. Back there. You touched on this a bit. Why is it that the european model is different than the american model and physics is physics. Hell does one become better vcf at what is different. I think the answer is in the limitation of the model to show it isnt a module to module so there are a lot of proclamations made and a lot of the differences in those approximations either with what is happening, but i think that the way that but i was surprised by is the importance of this observed weather in the data assimilation and how challenging it is to take the data youve collected and finthat iscollectt time and sort of tiny mans model scale and place in models able to insert the data and make it light up and a lot of what the advantage to the european model comes from the scheme actually now the director it was sort of her graduate research that started this particular way of winding up and correcting the weather inside the model and i think that sort of ability to constantly check the way the model is working and to be better at putting the model back on course with each step in time is one thintimeis one thing thad to. So, physics is physics, but the ability to actually observe the atmosphere is precise. I would add one more point on this. The differences between models arent that great. For years its been the most accurate in the world there is no denying that and i tell people if i only have tim had to look at one model or only had access to one model i would pick the european model because on balance it is the best, but not by a lot, and not in every situation. So one of the jobs they have to become skillfu skillful ad is understanding the bias of the model and when to use what model and in what situation. There are times when it has a better forecast in the european model. Thereve been hurricanes where they have better forecasts say you just have to look at the model collectively. There is not just the european and american. As a canadian, theres a uk model, models in japan and interNational Centers. We call this an ensemble approach like where do they have things in common and where are they forecasting alike, what are the differences and then you make a forecast of three to communicate the uncertainty based on where the models disagree. One way i like to look at how powerful they become is the models of the higher resolution than the reality which is to say we can say what the conditions are in far more places than we actually observe them. So they are where all of the data are we just have the actual observed weather is a small portion of the amount of detail the models are presented but it is a strange thing that they have higher resolution of the reality as it is observed. Im curious if there is a mac from the Academic Field that theres tons of researchers but not a lot of fun names. Can you see a lot of improvement coming from money or do you need more graduate students doing research . Where can those improvements be made . One of the things to me the shift that has happened in meteorology that is really abo about, the task of improving the model is a somewhat different skill set and when you see all of the action and the improvement of the system, it isnt clear to me if the programs are in terms of that and as with my other things, because the same, the skill sets of Software Design and the date of science, because there is such a pool on that for the Silicon Valley is a bit of a drag as well. He emphasized the need for them to have obviously a strong pass Computer Scientist scienti, Software Engineers as they continue to improve the american modeling system so you do need physical scientists and they rely on observations and data assimilation to keep improving that say youve got to keep adding observations in the system and making sure the observations are high quality and so in the model of physics you need them involved in that and attesting to model and finding where the model isnt working correctly and Publishing Research on how to better represent a certain atmosphere and moving the resource into the operations which is actually the whole Research Operations questions in the European Center it has been far superior to the u. S. Approach. And that is one of the reasons people who study this issue have concluded because they have better collaboration and research to operations and they spend a lot of time and effort on the research, so that i think that they recognize that and hes trying to beef up the research to operations within thoperations within theweather e modeling under. So, they can continue to make progress. I would be a little more critical of that. While there is talk of improving but its a step in the right direction, the institutional drag is enormous. It is the entire structure of the modeling system. Ive gotten sort of convoluted over the years. When you look at us or not clarity not only of the European Center for also of the uk office of the uk national Weather Service, if you recognize that we dont have at the moment frm the bureaucratic standpoint, that isnt into the first effort its a very small step. I can speak more freely about that. The motivation with the modeling initiative in the u. S. Is to move the system into the cloud and allow researchers at the universities to run a modele model in parallel and test things out and send improvements and identify airburst and send those back within this new modeling system so that they can improve the model so that is what they are trying to do. Time will tell whether it is successful. Its to try to get the american model at least so that is the goal whether they can accomplish it in a year and a half or five years and a half. We will see. Getting the blip on the phone this morning regarding the flooding, and basically do you see it moving just towards that model completely . I googled and found an article you wrote about how they revitalize them and i dont know if that fell by the wayside or something that still goes on like local Weather Forecast on the Channel Seven news or whatever. Has that kind of fallen by the wayside or are we moving to something completely different or does that pulled o that pullk up a there is a chapter in book when you talk about the technology as consumers use to get Weather Forecasts i think the way that information is received and changing and you are talking about weather, sports, news, and increasingly, people are overwhelmingly receiving Weather Information on their mobile devices. Whether you are talking about a Smartphone Application or the mobile web, whatever. So, yes i think absolutely that is one of the reasons we have been successful because we have been Digital First since he started providing an interactive platform for providing Weather Information people can only receive Weather Information but send stuff back to ask questions and have access to experts. You Program Certain cities you are interested in and its right there for you when you wake up in the morning. And when the weather is highly impactful, you get the alerts. I think one of the reasons one of the things the Smartphone Applications dont do which there is a need for is this human interpretation and icon with a cloud that doesnt tell you that much especially when the stakes in the forecast are high you dont understand what the possibilities are or how this will impact me. So if you want a more detailed level of information, you do need people to whatever platform you prefer you need that information presented to you in that way and some of that they dont do that for you so there is his decision process. I wish that we had visiting new york but truly it is that the moments where there is a lag between what the forecast is particularly the models are able to present. Stop raining at this time, and no way in which this kind of the high and low very deliberate hedging i think that is quite clear. Sometimes the forecast is very confident, sometimes its not and sometimes it is in my brain and it could be a good forecast but its not what you want to hear. But the system is technically spitting out and how we are willing to trust thagoing to tre process. You make up for that by presenting with a range of possibilities are in the forecast. How do you look at the forecast now since you have done the book versus the way that you datedid it before quick its quite interesting. Im not a trained as a meteorologist coming and i recognize what the limits are, i think that one of the things ive learned is by knowing the rhythm of the models, i can sort of see what trends are eve the n in the applications. So, the Weather Underground for example has the most granular detail in what it is presenting. And if you know that it is essentially being updated every 12 hours, you begin to recognize what the changes are happening in the forecast. I did an experiment of looking at the forecast for the publication date of my book each day and presenting it and it was amazing to see how little its changed over the days. If was aided the date she was 50 covering basically eight days, but sure enough the day came and it was rain and it didnt rain. It was a bit of both. For me, it is both trusting the percentage is more to recognize and 20 , 60 , and not just looking at when the ambrogi flips over and also recognizing that its long. I just want to make a point by the second year the overriding theme that makes it Crystal Clear and we will give you a better appreciation for the Weather Forecast and the forecasting whether it is someone on tv, someone on the radio we are destined in interpreting the information we are just translators. The work which was done to get the predictions where it is today is performed by the brilliant mathematicians and physicists in the world and is incredibly complex. It took decades of just incredible hard work. For the public, i dont think they get, i think the book makes this really clear how sophisticated and complicated and how rigorous the science is. To get the models that we have these days, the pretty faces you see on tv. Its how we got to where we are today. I would just say its amazed me that the goal is to stay out of harms way. Its not the case at all and it was amazing to see the kind of counter narrative that is made by people and requires the sort of human interpretation particularly as it becomes more dramatic. A complicated subject as with your book on the internet, youve made it accessible and readable and i would encourage everybody to buy a copy