How much does work environment matter? How deliberate should and can environment design be? There seems to be some consensus on the radically increased availability of distractions in modern life compared to a monk-style existence hundreds of years ago. Rather than to mentally exhaust ourselves with conscious and disciplined decisions against distractive actions, like picking up the phone or spending time on Twitter, we can try to design our environment in such a way that these distractions do not exist in the first place. The interesting question would be to understand if this changes our "baseline" in a meaningful manner, i.e. if we do not have access to these distractions for a long enough time, maybe we do not miss them anymore and hard work feels more stimulating.

Monasteries are places built to void such distractions. However, it can be difficult to stay fully sovereign in how time is spent when visiting a monastery today (religious practices). There is probably some optimal balance between how a day has some forced structure (e.g. meals at specific times) while maintaining freedom in what is being worked on. A counterintuitive but possibly fitting place might be a container ship. Days are highly structured, chores do not exist (food is provided by one central kitchen etc.), high-bandwidth internet does not exist and so on. To test this I embarked on a mid-size cargo vessel for 10 days. I downloaded a number of Maths textbooks and worked on them every day and made far more progress than I could have made not being isolated on a ship. Social life can be an issue, depending on one's personal dependence on such, but overall I would argue that a cargo ship journey provides many aspects of what available working environments often lack. Viability is obviously heavily restricted by the constraints set on types of work possible, i.e. work has to be possible without internet or communication in general. However, for tasks like self-study, this setup seems to be close to optimal.

Hypothetical scenario: in 30 minutes you will be told a 30-digit number. Immediately after seeing it, you will be put into a 3 months lasting artificial coma. If you wake up and can remember the number you will stay alive, otherwise you will be shot. How would you spend the 30 minutes maximising the probability of successful recall after the coma?

But also in less hypothetical settings, the role of memorization in learning and in the integration of new material is a crucial one. The consolidation of perceived information not only makes a piece of information available as a fact to be recalled but also changes how we think. But what factors influence the successful consolidation of perceived information? Most people are aware of spaced repetition tools like Anki to iteratively increase the halftime of a specific piece of information. However, the deliberate practice of recall is not the only way of controlling the forgetting curve of perceived information and the learned.

Many different factors influence how important the brain perceives a piece of information to be. The brain models this importance and encodes that piece of information with a strength proportional to that modelled importance. This heuristic seems to be a function of the current brain state in terms of different levels of neurotransmitter concentrations (or maybe not a function of, but rather it is itself the current brain state). These neurotransmitter and neuromodulator concentrations correspond to a range of psychological and cognitive states (conscious or subconscious): intensity of attention, arousal, emotion, level of metacognition, perceived novelty, perceived (reward) relevance. All these dimensions form a space of all possible combinations and resulting mental states.

The naturally arising question is then: what is the optimal position in this space, that as a mental state maximises later recall probability of learned information? I.e. we can view this space as the domain of an optimization problem, or through the lens of control theory, we ask how we can stay in a specific area of that space and more importantly, how we can move our mental state in that space to more desirable regions. Before we look at the individual dimensions of this space that seem to cover as much as possible of the variance in recall probabilities at test time, it is important to mention that it is very difficult to disentangle the effect of mental-state aspects (e.g. the current estimate of novelty in our perception) on cognitive performance in general versus on just memory formation and consolidation. At the same time this disentanglement seems rather unimportant given that due to the nature of cognitive performance, an increase of it in general will likely influence memory as well. A second remark on the factors that seem to influence memory consolidation the most: for some dimensions, it is hard to determine if they are partly contained within each other. E.g. the level of perceived novelty might directly influence memory consolidation, at the same, it is possible that it does that only through increasing the level of attention and thus would increase memory performance only indirectly. However, ultimately it is unimportant if memory performance is improved directly or indirectly.

Dimensions:

• attention: attention is the main mechanism of the brain to differentiate between important and unimportant stimuli and has (maybe the most obvious) effect on memory encoding and later recall ability.
• general arousal: arousal refers to the general state of psychological and physiological activation and is majorly defined through the interplay of the sympathetic (fight or flight response) and parasympathetic (rest and digest) parts of the autonomic nervous system. For cognitively non-demanding tasks a higher level of arousal is associated with higher cognitive performance, for cognitively more demanding tasks there seems to be a sweet spot of moderate arousal. The psychologists Robert M. Yerkes and John Dillingham Dodson formalised this as the Yerkes–Dodson law already in the year 1908. Arousal together with strong emotions also seems to be the main driver for so-called flashbulb memories, i.e. memories that are vividly recallable decades without deliberate practice and are mostly formed during strongly moving moments like childbirths or events of trauma.
• emotion: different emotional states, especially states of strong emotion, seem to have a strong effect on later recall ability. Likely strongly connected to the impact of general arousal and is highly associated with flashbulb memories as well.
• metacognition: according to Wikipedia "an awareness of one's thought processes and an understanding of the patterns behind them.". The reflection of one's own thought and reasoning processes might be most prominent in various meditation paradigms like mindfulness. Different studies show quite a significant impact on different measures of cognitive performance through metacognition (or more predominantly meditation). This includes the general practice of meditation and the practice of meditating immediately before a learning task. Again, it seems to be difficult to separate the effects on general cognitive performance and memory. Even more importantly, it seems to be rather challenging to separate the effects of meditation or metacognition on cognitive performance from those on attention and thus only indirectly on cognitive performance.
• perceived novelty: different cognitive frameworks like predictive coding are based on principles of prediction error and model updating. To generalise well and to not overfit on specific stimuli, it would be useful to model the novelty of stimuli and associate encoding and consolidation strength with such novelty. Several neurotransmitters, most importantly Dopamine and Acetylcholine are associated with this perception of novelty and memory encoding strength at the same time. Studies show that novel environments improve memory performance. The role of acetylcholine in memory and novelty perception is established. Artificially increasing dopamine levels through administering a dopamine precursor (L-DOPA) increases memory performance.
• perceived (reward) relevance: different neurotransmitters, most importantly Dopamine, are connected to the reward-importance modelling of stimuli in the human brain. Research shows that this perceived reward relevance (also called incentive salience) is correlated with memory performance.
• context similarity train and test time: so far we have treated the impact of different dimensions like attention as being separate between train and test time. However, there is not only the notion of conditions being optimal for train and test time independently, but also the orthogonal effect of conditions just being similar between train and test time. Research shows that the similarity of specific aspects of the mental state (including the current perception of external stimuli aka the environment) between train and test time has a strong effect on successful recall probability. It very much feels like a form of compression in which a piece of information is stored not in its full size but conditioned on the current context. If my environment is represented through 1 bit and I want to store a piece of information of 10 bits, I only need 9 bits for storage if I condition the information on the environment and (obviously heavily constrained by only then being able to retrieve the information when the environment is fully reinstated). If the environment present during encoding has been memorized before anyway, it can also be mentally reinstated during a recall attempt, removing the condition of being in the same context physically. People have made use of this dynamic for thousands of years through the so-called method of loci in which pieces of information are associated with physical locations in an imagined memory palace. The abilities of mental reinstatement seem to be trainable and reach vast areas from visuals, and sounds but also physical movements and emotions.

Moving in that space?

Coming back to the question from the beginning, how can then recall probability be truly maximised? The different dimensions outlined above already indicate an optimal subspace of the space of mental states. The more interesting question is, how can we move in this space, that is how can we consciously influence all of the different aspects affecting recall probability? This task is a very hard one, firstly due to how many of these aspects are related to subconscious processes making them hard/impossible to control and secondly due to how difficult it is to measure a mental state on these dimensions, which if possible would make the self-training of moving in that space a lot easier. It also presents a difficult trade-off: is time better spent moving in this space (and learning how to move in this space) or doing the actual learning task? While this remains an open question, I will outline some thoughts on possible avenues of approaching mental-state optimality.

• attention: probably best trained and improved through attention exercises and focused meditation (i.e. meditation pratices that involve focusing on a single point or stimuli for prolonged periods of time)
• general arousal: specific types of meditation allow to increase and control levels of stimulation (the legendary Tummo meditation would be an example). Potentially more practical interventions might be the use of different breathing techniques and (light) physical activity.
• emotion: music?
• metacognition: probably best trained through performing deliberately performing metacognition, often central to meditation practices like mindfulness
• perceived novelty: could be achieved through performing a learning task in novel environments and settings and through deliberately trying to "defamiliarize" oneself with material and concepts learned
• perceived reward relevance: potentially achieved through the implementation of short-term rewards (gamification, treats). Some research suggests the visualisation of "mastery" to be helpful. Additionally, it might be helpful to frame material in ways that imply personal connection and evoke a perception of wonder. I.e. making oneself aware of how truly fascinating but also mysterios the nature of a specific problem is, might increase recall probabilities of associated information.
• context similarity: if possible true physical similarity between train time and test time environment. Otherwise: method of loci to an extreme. I.e. not only imagining a specific scene to be associated to a memory but also a specific emotion, proprioceptive state, sounds etc., again ideally associations present during test time anyway to limit the need of mental reinstatement.

What is statistical learning theory? Statistical learning theory asks sort of fundamental questions: what can be learned? what cannot be learned no matter how hard we try? We seek to find answers to these questions by formulating learning as the computation that a learning algorithm performs. A learning algorithm is a function that maps data to a hypothesis in some hypothesis space, which itself is a function space.

At this point it makes sense to take a moment to think: can every output of a learning system be understood as a hypothesis, i.e. a function?

For many processes of learning that might be true, especially the engineered ones. Take for example a neural network. The design, i.e. the architecture, of the neural network defines the hypothesis space, i.e. the different functions that can be learned. The learning algorithm now finds the best function, which is a function easily understood mathematically due to the clear dichotomy between inputs and outputs. But what about something like a brain? There seems to be a gap between the ease with which ML algorithms can be understood as hypotheses and the difficulty of understanding the brain as one mathematical function. Or actually, it is easy to view the brain as a function, but it is harder to understand the learning algorithm at hand. There is no empirical risk minimization trying to "fit" the function mapping perception to actions to some "true" version of how a brain should behave. It is rather a beautiful interplay between the search process of evolution, finding brain configurations that reproduce and persist, i.e. slow-scale adaption, and learning processes, i.e. fast-scale adaption.

Germany's dependence on natural gas is undeniable, with consumption of this fossil fuel rising steadily over the past decades. Even amidst the shift towards renewable energy sources, natural gas continues to play a crucial role, primarily as a transition fuel, enabling organisations and policymakers to meet short-term CO2 reduction targets. It's also critical for heat-intensive industrial processes and residential heating, areas that have traditionally been difficult to transition to clean energy sources. Given the unique position of natural gas within our economic system, understanding and forecasting its demand is of paramount importance, particularly in times of supply disruption, such as during the Ukrainian war. Proper planning and procurement of natural gas supplies is essential to prevent severe shortages that could drastically affect a country's gross domestic product.

Attempts to forecast natural gas consumption have a long history, with early models focusing on statistical approaches that took into account aspects such as household income, GDP and ambient temperature. The advent of advanced computing resources led to the application of more sophisticated models, including Artificial Neural Networks (ANNs), Support Vector Regression (SVR) and deep learning models such as Long Short-Term Memory (LSTM) models. While these forecasting techniques have led to unprecedented advances, much of the previous work on natural gas consumption modelling lacks reproducibility due to limited access to the underlying data or exact model parameters. This is often due to licensing restrictions imposed on authors.

This project aims to bridge this gap by providing an open source and publicly available platform for natural gas consumption forecasting in Germany. The focus is on ensuring that critical data on the dependent variable and potential independent variables are publicly available, that forecasts are implemented in a time-efficient manner using state-of-the-art models and frameworks, and that these forecasts are continuously published and productized. The aim is to make energy-related scientific issues, which have a significant impact on everyday life, more accessible to the general public.

Finding the right data for a model can be challenging. Dependent variables for the model have been established in literature. Recent works offer an overview of factors influencing natural gas consumption in today's energy market. A study of literature surveys and discussions was conducted to identify consumption indicators.

Special challenges are faced in this work. All datasets used must be open-source and programmatically accessible for the model.

The potential regressors considered for the model are:

• Temperature: The relationship between natural gas consumption and temperature is well-established in literature.
• Natural gas prices: The influence of pricing on consumption is well-studied, described through "price elasticity."
• Crude oil prices: Prices of other energy commodities can impact natural gas consumption due to substitution effects.
• Coal prices: Wholesale prices of coal can also affect natural gas consumption.
• Electricity prices: Electricity is an energy commodity that can substitute natural gas to some extent.
• Auction prices of European Emission Allowances (EUAs): The European Emission Trading System caps natural gas consumption, but the relationship can be both positive and negative.
• Natural gas storage levels: Storage levels of natural gas can serve as an indicator of demand and consumption.

To obtain the necessary data, various sources are used. Consumption data for natural gas in Germany is provided by Trading Hub Europe GmbH. Temperature data is sourced from the atmospheric reanalysis model ERA-5 and real-time weather information from Open-meteo. Natural gas prices are obtained from Trading Hub Europe GmbH. Brent crude oil prices are available from the U.S. Energy Information Administration. Electricity prices data is provided by Ember. Auction prices of European Emission Allowances (EUAs) are obtained from the European Energy Exchange AG (EEX). Natural gas storage levels data is sourced from Engie.

The data analysis reveals seasonal patterns in natural gas consumption and the importance of temperature as a factor. The historic data of consumption, temperature, and other variables are examined in the figures provided.

After understanding the dynamics of natural gas consumption, a model was designed, trained, and evaluated to forecast Germany's natural gas consumption. Previous models, such as a simple seasonal model and a piecewise-linear temperature model, showed promising results, but more sophisticated models are now established in literature. Several models are discussed, including multiple linear regression, Autoregressive Exogenous Model (ARX), Seasonal Autoregressive Integrated Moving Average Model with exogenous regressors (SARIMAX), XGBoost, and NeuralProphet.

The performance of these models is compared using metrics like Mean Absolute Percentage Error (MAPE). The temperature-only versions of ARX and NeuralProphet are also evaluated to create a fully automated forecasting system. The NeuralProphet model performs well in both short-term and long-term forecasts. A hybrid approach is proposed for temperature forecasting, combining short-term forecasts from weather services with a mid-term model trained on historic data. The combined temperature forecasts, along with the temperature-only NeuralProphet model, enable the forecasting of natural gas consumption for the next 365 days.

This approach strikes a balance between predictive performance and deployability, making it suitable for practical applications.

The primary aim of this project was to develop and deploy a public-facing natural gas forecasting model, which re-trains and updates predictions daily. The resultant data needs to be stored and served to the public in a straightforward manner. In terms of architecture, the design leans towards simplicity and efficiency. Both temperature and natural gas consumption prediction models are encapsulated in a Docker container as Python code. The results are saved using a storage service that channels forecasts via a backend to be accessed by a frontend. This entire backend operation is hosted on Google Cloud Platform.

A scheduling service is set up to activate daily model re-training and run the model at a certain time. The new forecasts are stored in Google Cloud Storage, from where they can be accessed by users through the frontend. The frontend comprises a web application built with TypeScript and the Next.js framework, with Tailwind CSS aiding in styling. The app's primary function is to visualize data retrieved from the storage endpoint on Google Cloud Storage, avoiding the need for implementing any additional logic.

While the model re-training and runs take place on the Docker container hosted on Google Cloud Platform, the frontend is also hosted on a fully managed platform, Vercel, to reduce costs and administrative tasks. Vercel, which offers free service for non-commercial use, links to a Git repository of a web application built with one of many approved frameworks, managing the deployment and serving of the web application. Though utilizing fully-managed infrastructure solutions may not be a necessity or suit everyone's unique requirements, in this instance, it helps to keep the solution simple and easily reproducible.