How can we distinguish between good and poor scientific answers?

007_Post

The awareness of science differs between a scientist and the general public. A scientist tries to understand everything down to the smallest detail and the general public likes to know little detail about everything. How can everyone be satisfied in a good way?

We regard, often unknowingly, science somehow as a holy oracle which can provide straight forward simple answers to any given question at once. In contrast, a scientist likes to understand how the oracle came to an answer, but is not much interested in the answer itself.

This different awareness of science is difficult to resolve. How can we decide whether a scientific answer is reliable and not an oracle whisper out of nowhere or based on only limited knowledge?

In my experience four mechanisms can be an indication of poor science answers:

  1. On the edge of extinction

    If a research result is apocalyptic and forces to act quickly you should be suspicious. This type of science answers is like a door-to-door salesman, who tries to sell you something less worth than he makes it appear. Think about how long it takes to get reliable scientific results. It takes month to years and even decades to find good answer and therefore there can be/is no rush to act quickly. This will only force you to stop thinking and to act in the authors’ fashion.

    Apocalyptic science results are usually guided by faith or fear and the conclusion is often drawn before the facts are known. Fear and faith are not good advisers to give the right answer, not even outside of science.

  2. Repetition

    If you read and hear the same scientific answers and arguments many times then most likely, in the good case, the authors try to make themselves important and, in the bad case, they try to make something suspicious to look correct and proven. Either way, nothing becomes true only because someone repeats a statement many enough times.

  3. Political involvement

Usually, if politicians are involved in scientific answers, public opinion is more important than facts. The purpose of a politician is to channel public needs and wishes into state rules and laws, but they don’t have the knowledge to provide scientific answers. Politicians only support scientific answers which suits their own ambitions.

  1. Lobby-work

    Everyone has an agenda, but this does not necessarily lead to poor scientific answers. Nevertheless, if a research group reaches a certain degree of power an agenda becomes lobby-work with their own dynamic. Unfortunately, scientific lobby-groups try to keep alive or in power even when new research results point in a different direction. They become obsessed and ignore all results which challenge their built paradigm by all costs necessary.

As Ibn Al-Haytham an ancient scholar wrote 1000 years ago: “We must be seekers of truth and not rely on any consensus.”

“Research should always refuse authorities as such. Skepticism is the highest of researchers duty; blind faith is the one unpardonable sin”, as T.H. Huxley felicitous pointed out.

Advertisements

Thoughts on oral presentations

TitleFigure

First published on LinkedIn, February 26, 2017

Last December I attended the AGU fall meeting to see and learn from the broad spectrum of Earth sciences presented on this conference. As a geophysicist with seismic data processing and engineering as background I was curious to see what other scientists are engaged in and I believe that the AGU is the right place for that.

So I started with a fresh mind and I had a tough schedule like everyone else on this conference. However, after two days following presentations from different fields of expertise I started to lose focus and it became more and more difficult to keep track. I started to wonder why? Was it simply too much information I tried to take in? Or was there something else draining my attention?

Because this feeling is nothing new, the same has happened to me on other conferences before. But on the AGU fall meeting I got an idea why. I started a very rough investigation parallel to my notes from each presentation I followed. In order to keep my little research simple, I separated the presentation figures into two different categories. The first one I called non-multimedia such as diagrams, tables or maps and the second category I called multimedia such as pictures, videos, animations or self-explaining illustrations. Then I simply counted how many figures of each of the two categories were shown in the presentations.

The results I got surprised me in their clearness: On average, each presentation showed 23 non-multimedia and only 3 multimedia figures. This average was consistent for different sessions and disciplines as well. In other words: in every session with 7 presentations, we had to focus on more than 160 diagrams, tables and maps! For a non-specialist in a specific field this represents a real challenge that is hard to fulfill. I have not judged the quality of each figure, but it became obvious to me that the presenters were more focused on showing a constrained line of their results than to make it easy to understand. There was one presentation with 49 non-multimedia figures, and another one showed 10 diagrams on a single slide. These are extreme outliers, but I think you get my point.

Maybe it is time to think about a different strategy for how we should present our results. 30 years ago, when computer based presentations became popular it was difficult to show videos or animations, but the same old rules on how one should present a scientific topic are still very much alive today.

There is nothing wrong in showing diagrams to prove the presented results, but it is not how the audience keeps focus; except maybe some specialist who know exactly what the diagram means. But most of the time we are not specialists on every single presentation in the session we are in. Ask yourself: is there any presentation from that you could remember the data that were presented after a couple of days? Most likely not. But if so, I guess you remember a video, animation or an excellent illustration, not the diagrams.

I would like to propose a different way to present results in an oral presentation. Take a maximum of only three of your most important diagrams from your results and build an illustrated story around them. I believe it is more important to take the audience on a trip through your results than to show a consistent chain about how you got from the data to your results. Consistency is important when you are writing an article, but in an oral presentation the story around it is what we remember. Not every scientist is a good story teller, but if you start to tell why you have done your investigation and what your motivation was the audience will be more focused. Use pictures and (animated) illustrations to show how you answered your question which drove you into your investigation.

I am aware that I am not writing something new, but as my little investigation points out we can still improve our success to share ideas and knowledge and, most importantly, enhance the chance it is remembered. Instead of having 160 diagrams in one session, we would then have only 21 diagrams but 7 very interesting stories attached to them.

Why we lost the atmosphere of constructive debates

005_ConstructiveDebates

In my previous post I discussed the problem of overheated and narrow-minded debates between two opposite opinions [here]. And one thing was really clear for me: There was no atmosphere of constructive debate. How can this be? Two scientists discussed an issue and it became personal instead of focusing on what both sides have to tell. In fact, the outcome of this debate has a huge impact on science in specific and the society in general, so it should remain objective. But this happens not only to scientific debates; companies experience the same behavior in their working teams and managers have to solve personal issues between colleagues on a regular basis instead of focusing on the problems. In my personal experience the source of the problem is two-fold:

  1. The opponents are at separate locations, often without direct personal contact

In our information based society emails, social networks and blog posts are the main media for debates. This is useful to some extent, because writing something down clears your mind and you work on it to organize your thoughts. Nevertheless, no matter how accurate you write your sentences each reader will interpret them differently. The human nature is optimized to interpret information from their own perspective. Therefore a growing conflict is inevitable if we focus only on writing!

In a verbal communication, like face-to-face, the opponents provide redundancy in form of body language, voice control and empathy. This can help to reach common ground and solve conflicts before they start. Unfortunately though, communication on conferences and workshops as well as in many private sector working teams are usually not a place for constructive debates. The sessions or teams mostly gather people with the same opinion or at least with the same understanding or background.

A couple of years ago I attended a CO2 storage workshop in Berlin and most of the attendees were climate change advocates. When I told them that I was skeptical they rose their eyebrows and changed the topic of the conversation, instead of engaging in a constructive debate exchanging arguments.

  1. Avoiding disagreements

In the old days there was a patriarch or boss who said what everyone under his authority had to do. There was no debate at all. During the last several decades we developed a new approach on how we should debate. This approach changed companies and research organizations to what is called a flat hierarchy, where we now have managers and supervisors who inspire and take care of the team needs. Their duty is also to work with conflicts which is great, but we lost something on the way that we actually need. We forgot to debate in a constructive way. Instead of solving conflicts we learned to be nice to everyone and not to offend someone.

However, sometimes it is important not to be nice! In order to solve a specific problem you need conflict in the team, because you will never reach the bottom of the problem at hand if not all opinions are heard and discussed. Controversial opinions are always important to evaluate all aspects and risks of a given problem. A heated discussion can be very fruitful – as long as the attendees of the discussion can go to the pub afterwards and have fun together.

Conflicts are not per se bad, but should not be taken personally or used to humiliate others. A constructive debate should be built towards a conclusion. However, some arguments don’t necessarily get resolved during one meeting and should be discussed again at least one day later.

Did we ever have a period of constructive debate, you wonder? Usually only when something new is starting, like a new research or industrial project, newly founded company where no deep ties, structures or relations exist a climate of constructive debate is found.

As a personal example, I worked for a CTO once who was so afraid to have conflicts during his meetings that he basically ignored all project red flags given from his project managers. There was no debate and no climate where subordinates could thrive. The meetings deteriorated to nice small talks without any decisions or outcome. Because of this behavior the company had to resign two major projects and lost a big costumer.

In order to avoid conflicts some managers try to trivialize problems or assemble teams with the same background, culture, nationality or beliefs. But this in my opinion is absolutely wrong!

A team should always be assembled with different personalities, opinions and experiences. The more diverse the team is, the more creative and successful will be the outcome. Some of the team members should be creative and some should do the pedestrian work, but everyone in the team should be considered as equal. This is of course more work load for the manager who has to provide guidance, but the reward will be astonishing.

Our new living and working environment provides us with countless opportunities, but with some new challenges as well. We can use our opportunities in a good way if we remember that:

– It is essential to have direct personal contact to reduce the risk of misunderstandings.

– Constructive conflicts are essential for the success in scientific and industrial projects.

4 reasons why climate change is controversial

004_ClimateChangeControversial2

First published on LinkedIn, 28.07.2015

I am following the ongoing discussion between climate change advocates and skeptics since my studies in geophysics and meteorology at the University of Hamburg where one of the advocate group is located. I will not point fingers to one side or the other because both sides provide useful insights to solve the question whether anthropogenic climate change is real or not.

But since my study years the discussions grew from scientific disputes to religious-like beliefs and the discussions have reached a level of personal opinions rather than facts.

Let me illustrate this on a recently published dispute between Marotzke and Forster [1] as climate change advocates and Nicholas Lewis as climate change skeptic [2]. It contains three reasons for the controversy around climate change. But I start with a fundamental issue, which is only implicit part of the Dispute.

  1. Probable flaw in the main assumption

Correct modelling of the past don’t necessarily mean that the same model yields correct future predictions. Geological processes like plate tectonics, mountain building (orogeny) and meteorological processes like the water (hydrological) cycle happen in the same manner since the beginning of the earth until now and therefore we can predict future behavior by understanding the past. But the Earth‘s climate is a different category. It is known as a chaotic process which means that small changes in one parameter can cause large effects in the result (e.g. butterfly effect). In fact, some of the most popular chaotic figures, the Lorenz attractor, was discovered by Edward N. Lorenz a meteorologist by analyzing meteorological data and the underlying mathematics. He claims in his paper [3] that: “If the [weather] system is stable, its future development will then remain arbitrarily close to its past history, and will be quasi-periodic. …since the atmosphere has not been observed to be periodic…..no forecasting scheme could have given correct result…” In addition to the usual chaotic behavior one large (VEI >= 5; [6]) volcanic eruption will make all existing anthropogenic climate change forecasts obsolete for decades: And in the past 25 years there were 2 eruptions with VEI >= 5; [7].

  1. Simulation driven approach

Most of the anthropogenic climate change forecasts are model based rather than data driven and are built from a positive assumption. I will not go into the mathematics, but I have to show one equation to make my point. The following equation is the main subject of the discussion between Marotzke&Foster and Lewis where an energy balance is described as: ΔT = ΔF / (α + κ) + ε. Don’t worry, it is not necessary to understand the equation in detail. You just need to know that three of the four quantities ΔF, α, κ are parameters modelled from simulations and used to simulate ΔT due to lack of direct measurements. You can see from the quantities‘ names that they have no direct relation to known physical quantities: ΔF = change of effective radiative forcing, α = climate feedback parameter, κ = ratio of change in the heat uptake of a climate system. They are hypothetical constructs. The fourth quantity ε is a computer generated random value.

In short: they use simulation results to simulate something that cannot be measured directly. I am not saying you can’t do such thing, but it becomes more and more difficult to understand the reliability of the results, especially if at least one simulation is nonlinear or even chaotic. It doesn’t help to claim that the prediction is correct because 114 simulations based on the same model / assumptions provide the same results [1]. This is comparable to: 114 translations of Grimm’s fairy tales are similar, so the story must be true.

 

  1. Opinion and agendas before facts

Science has nothing to do with opinions and agendas. But we scientists, as social beings, have something to do with it and we have to work with that every day. Unfortunately, some scientists become believers of their own opinion instead to discuss issues in a constructive scientific way.

Marotzke is already convinced that the climate change is anthropogenic and he said [4]: “Sceptics who still doubt anthropogenic climate change have now been stripped of one of their last-ditch arguments”. This is an opinion not a fact!

Due to the nature of climate change the chosen approach is an inductive projection which means we don’t know exactly the mechanism behind climate change. We can’t even be sure that anthropogenic climate change exists. Measuring a temperature increase in the last hundred years and a change of our daily weather behavior compared to the last century hasn’t necessary something to do with an increase of CO2 or that it is manmade. There are other explanations that are equally probable. Maybe we have lived in a lucky period where the climate was quasi periodic and this starts to change now, independent of what we are doing?

  1. Missing constructive discussions

The concept of science is to find reliable and reproducible descriptions of observations we make. In other words: Is there something we don’t understand we try to find a scientific explanation. Due to our human nature we get sometimes lost in our hypothesis. Therefore we should embrace questions which contradict our hypothesis and should discuss them constructively.

Unfortunately, the opposite is quite often the case. I think Lewis provided a good mathematical explanation why the results from Marotzke and Forster don’t work. But he finished his analysis with following sentences: “The statistical methods used in the paper are so bad as to merit use in a class on how not to do applied statistics.” and “All this paper demonstrates is that climate scientists should take some basic courses in statistics and Nature should get some competent referees.” Writing these statements doesn’t help to convince the other side. But this shows how overheated the discussion already is and that the sceptics feel not been taken seriously.

Unfortunately, Marotzke and Forster are not taking Lewis analysis serious. They basically repeat in their answer what they already have published in Nature [5]. And the main reason for why Lewis wrote his analysis was rejected with following statement: “It has been alleged that in [1] we applied circular logic. This allegation is incorrect.” This shows the narrow-mindedness and arrogance of the advocates of climate change. To have the majority doesn’t mean they own the truth.

In order to solve the climate change issue advocates and sceptics should work together as equal partners in the same group or directly within the IPCC. Only if both sides are satisfied with the results we might be getting closer to the truth.

[1] Jochem Marotzke & Piers M. Forster. Forcing, feedback and internal variability in global temperature trends. Nature, 517, 565–570 (2015)

[2] http://climateaudit.org/2015/02/05/marotzke-and-forsters-circular-attribution-of-cmip5-intermodel-warming-differences/

[3] http://dx.doi.org/10.1175/1520-0469(1963)020<0130:DNF>2.0.CO;2

[4] http://www.mpg.de/8925360/climate-change-global-warming-slowdown

[5] http://www.skepticalscience.com/marotzke-foster-respond-to-lewis.html

[6] https://en.wikipedia.org/wiki/Volcanic_explosivity_index

[7] https://en.wikipedia.org/wiki/List_of_large_volcanic_eruptions_of_the_20th_century

Is science apocalyptic?

002_ScienceApocalipticFirst published on LinkedIn, 15.06.2015

This might be a strange question, because science is rational, analytic and always objective. Isn’t it?

But if you follow the media you find articles where authors claim that scientists say the apocalypse is near if we do not act quickly.

 

Apocalyptic scenarios

In the 1950s it was the atomic bomb and the use of atomic energy which some scientists believed may cause the “Weltenbrand”, and in the 1970s and early 1980s a new ice age was thought to freeze the northern world (e.g. Sir Fred Hoyle, 1981 in his book “Ice: How the new ice age will come and how we can prevent it” and his solution was to warm the oceans) and it was important to act quickly. In the 1980s and 1990s the hole in the ozone layer was a nearly hysterically discussed topic. Some call this a successful story and that the hysteria helped to solve the problem by preventing CFC in modern products through international agreements on reducing the consumption of ozone-destroying chemicals. But the panic was greater than the actual effect on earth, and the hole in the ozone layer does still exists! In my opinion some questions were not answered, such as how long the hole in the ozone layer existed prior its discovery in the 1980s, and whether CFC was its only cause or if there was something else affecting it that we didn’t know. Didn’t matter, we had to act quickly. Right after the hole in the ozone layer the discussions about climate change started again, but this time in the opposite direction; not cooling, now warming. And before substantial facts were available the conclusion was already made: the warming is anthropogenic and the end of the world is close. We have to act quickly! In the 1990s and right now a pandemic virus outbreak is at our doorstep and we have to act quickly, yet again.

 

Always the same mechanism

There are more apocalyptic scenarios, but I think you have got my point that we had to fear a number of apocalyptic events over the last century. Nevertheless, none of this events happened yet as they were predicted. If you follow the news you will usually not get to the bottom of the real problem, the only thing you get are opinions. There is always crucial knowledge or information missing and the mechanism as how scientific research meets the public is always the same: First one scientist discovers a possible threat to the world or life on earth as we know it, then media inflates the discovery to an apocalyptic scenario and after lots of repetition we start to believe it without questioning. And we have to act quickly, of course!

 

Fear and hasty decisions

Beside the problem that science might lose their reputation, this “common knowledge” will lead to wrong decisions, because it is based on fear and opinions rather than facts. One example: In the 19th century people stripped down all rods from their houses because they believed that lightning rods cause thunderstorms. The result was as you might expect: Lots of houses burned down after thunderstorms! Another example: In the 1990s there was a forest dieback and a culprit was found in sulfur dioxide from fossil energy. So the industry started to produce motor fuel and energy without sulfur. The sulfur-dioxide in the air decreased afterwards as the forest dieback. Now, in the light of assumed global warming, some start to discuss to inject sulfur-dioxide into the air and ignore more serious problems to our environment.

 

My point is: Scientific questions should not be used for hasty decisions which may cause fatal problems to our world or life.

Are computer simulations useful in all fields of science?

Computer simulations have a long history as scientific tool and started basically in meteorology and nuclear physics in the late 1940s. Since then, it has become essential in nearly all scientific fields. However, the greatest success computer simulations achieve is in engineering, due to the well-established theoretical knowledge and solvable differential equations. For example: new cars are already simulated up to 100% before production in order to eliminate flaws and to comprehend problems like material stress between component interfaces, optimal design to reduce aerodynamic drag and the interaction between engine and driving comfort.

 Successful tool in all science areas?

Computer simulations are a very successful tool for engineers where everyone can experience the results in daily life. But, how about computer simulations in science areas where we do not have all theoretical knowledge we need? In other words: would you buy a car based on unreliable simulation results?

There is a saying in the academic world:

A simulation is only as good as the theoretical knowledge from the real world.

But what does that really mean? Some questions rise immediately, that need to be answered:

  • How good is good?
  • How can one evaluate the reliability of the simulation output?
  • How good are the mathematical equations or the mathematical understanding?
  • How about limited or missing Information?
  • How stable is the simulation if one change some parameter values?
  • How can one be sure that all necessary theories are included or what are the minimum conditions we need?
  • How many simulation parameters do one need and are all simulation parameters tied together correctly?
  • How about cross effects between simulation parameters and the linked differential equations?

You might think: “That’s simple: Just compare the simulation output with real world measurements!” Only to realize after a short while: “Wait a moment, what if we can’t set some simulation parameters due to missing values or differential equations? Or what if we don’t understand the real world in the needed detail? Or how can we predict future behavior if we don’t really understand the past.”

And suddenly, we are back where we started: A simulation is only as good as the theoretical knowledge. Nevertheless, they have become popular in more and more science areas. In such a way that one might wonder: “Is a computer simulation useful in all fields of science?” My answer is:

Yes, but…”

In my experience, many simulations suffer from limited thoughts being put into answering the questions above. Although a number of philosophers discuss how to answer these questions (e.g. epistemology), this is not so often in the science community; at least not communicated. I don’t really know, why. Answering these questions is essential and should be part of each simulation result discussion. Answering these questions is often quite difficult and sometimes challenging, but nonetheless necessary! If we can’t answer those questions we should not rely on simulations.

I believe that simulations are a good scientific tool to estimate how well we understand the real world processes. From simulation results we can tell what kind of experiments and investigations we should undertake next to close knowledge gaps. By comparing simulation output with real world measurements we can establish a feedback loop which improves the simulation results and decrease the difficulty to answer the questions. After some cycles the simulation results might even be used to predict future real world behavior.

My take home message is:

We should not rely on simulation results only!