Weather and climate models in 16-bit arithmetics: Number formats, error mitigation and scope

13 The need for high precision calculations with 64-bit or 32-bit floating-point arithmetic 14 for weather and climate models is questioned. Lower precision numbers can accelerate 15 simulations and are increasingly supported by modern computing hardware. This pa16 per investigates the potential of 16-bit arithmetic when applied within a shallow water 17 model that serves as a medium complexity weather or climate application. There are sev18 eral 16-bit number formats that can potentially be used (IEEE half precision, BFloat16, 19 posits, integer and fixed-point). It is evident that a simple change to 16-bit arithmetic 20 will not be possible for complex weather and climate applications as it will degrade model 21 results by intolerable rounding errors that cause a stalling of model dynamics or model 22 instabilities. However, if the posit number format is used as an alternative to the stan23 dard floating-point numbers the model degradation can be reduced to a tolerable min24 imum. Furthermore, a number of mitigation methods, such as rescaling, reordering and 25 mixed-precision, are available to make model simulations resilient against a precision re26 duction. If mitigation methods are applied, 16-bit floating-point arithmetic can be used 27 successfully within the shallow water model. The results show the potential of 16-bit for28 mats for at least parts of complex weather and climate models where rounding errors 29 would be entirely masked by initial condition, model or discretization error. 30 Plain Language Summary 31 64-bit floating-point numbers are the standard number format for scientific com32 puting in fluid dynamics, which allows for very precise calculations with negligible round33 ing errors. The need for calculations at this precision level has been questioned for weather 34 and climate models, as errors are caused primarily by insufficient observations or defi35 ciencies of the models themselves. Reducing numerical precision can accelerate simula36 tions and low precision number formats are increasingly supported by modern comput37 ers. This paper investigates the potential of low numerical precision with numbers that 38 only use 16 bit of information, when applied within weather and climate applications. 39 There are several 16-bit number formats that can potentially be used, all of which have 40 considerably larger rounding errors than the standard 64-bit numbers. In this paper, the 41 different number formats are applied in a two-dimensional oceanic or atmospheric cir42 culation model. A simple change to 16-bits for all calculations will not be possible as it 43 will degrade simulation results. However, if mitigation methods are applied, 16-bit cal44

Software emulators for other number formats than Float32 and Float64 are often 88 used to investigate rounding errors caused by lower precision formats (Dawson & Düben,89 2017). Although emulators are considerably slower than hardware-accelerated formats, 90 they allow a scientific evaluation of the introduced errors without requiring specialised 91 hardware, such as FPGAs (Russell et al., 2017). Unfortunately, the computational per-92 formance cannot be assessed with software emulators. 93 Reducing the precision raises questions of the real bitwise information content. In 94 simplistic chaotic models only a minority of bits contain real information (Jeffress et al.,95 2017), providing an information theoretic argument for reduced precision calculations. 96 manuscript submitted to Journal of Advances in Modeling Earth Systems most applications. We will therefore focus on the discussion of the other number formats 136 in the rest of the study. (1) The exponent bits e are interpreted as unsigned integers, such that e − bias converts  Posit numbers arise from a projection of the real axis onto a circle (Fig. 1), with manuscript submitted to Journal of Advances in Modeling Earth Systems regimes into bits, posit numbers use regime bits which extend the standard on floatingpoint arithmetic. Regime bits are a sequence of identical bits after the sign bit, termi-164 nated by an opposite bit. As the number of regime bits is not fixed but flexible, the sig- 165 nificand increases in length for numbers towards ±1, when fewer regime bits are needed. 166 Consequently, a higher precision around one can be achieved with posits, which is traded 167 against a gradually lower precision for very large or very small numbers. number, except for zero and NaR, can be written as p = ±2 i · (1 + f ) for a given inte-178 ger i. In the following, a posit number format with n bits including e s exponent bits is 179 denoted as Posit(n,e s ). 180 We provide an example in the Posit(8,1)-system (i.e. useed = 4), which encodes 181 the number 14 as follows The sign bit is red, regime bits orange, the terminating regime bit brown, the exponent 183 bit blue and the fraction bits are black. The k-value follows from the number of regime 184 bits n r . For regime bits being 0 k = −n r , for regime bits being 1 k = n r − 1, in this 185 example k = 1. The exponent bits are interpreted as unsigned integer and the fraction 186 bits follow the IEEE floating-point standard for significant bits.

187
In order to use posits on a conventional processor we developed for the Julia pro-  decimal precision = − log 10 | log 10 ( x repr x exact )|. It is defined as the distance δ between 1 and the next largest representable number, and 206 can be given in terms of decimal precision as = − log 10 (log 10 (1 + δ 2 )).  In many models, it will not be possible (or useful) to use 16-bit arithmetic through-235 out the entire model. Some model components will be more sensitive to a reduction in η can be interpreted as pressure for the atmosphere (Gill, 1982). The shallow water sys- being the undisturbed depth, representing a meridional mountain ridge at x = Lx 2 span-309 ning from the southern to the northern boundary. The standard depth is H 0 = 500 m.
and vanishes at the boundaries. The water density is ρ = 1000 kg m −3 and F 0 = 0.12 Pa.

318
The wind forcing acts as a continuous input of large-scale kinetic energy that is balanced 319 by the dissipation term.

320
The dissipation term D is the sum

324
The shallow water equations are discretized using 2nd order centred finite differ- The dynamic range of representable numbers with 16-bit arithmetic is for most for-346 mats discussed here considerably smaller than with Float32 or 64 ( Fig. 2a and Table 1).

347
It is therefore important to rescale the calculations to limit the range of arithmetic re-348 sults to stay within the bounds of a given 16-bit format.     (Fig. 5a), which is typical for turbulent flows. A weak westward mean 421 flow is found at the northern and southern boundary. No 16-bit format was found to have 422 a significant impact on the mean state. The variability of the flow around its mean state 423 is high throughout the domain (Fig. 5b). The variability is significantly increased by 10 424 -30% with 16-bit arithmetic, especially with Posit(16,2). This is probably caused by round-425 ing errors that are triggering local perturbations which increase variability.

426
The turbulence in shallow water simulations is largely geostrophic, such that the Horizontal bars denote the mean, 10th and 90th-percentile in respective colours.
-20-manuscript submitted to Journal of Advances in Modeling Earth Systems u g can be derived from the sea surface height η as and deviations from the actual flow u are the ageostrophic velocity components u ag . We 430 project both components on the actual velocities to obtain the flow-parallel components 431ũ g andũ ag via The geostrophic velocities in the shallow water simulations can reach up to 2 m/s, are 433 hardly negative (i.e. against the flow) and have a mean of about 0.7 m/s (Fig. 6a). This bilities that lead to the observed increase in ageostrophy. We conclude that although the 445 geostrophic balance in the simulations is maintained, rounding errors lead, likely due to 446 an increase in ageostrophy, to a higher variability in the flow field.

447
As 16-bit arithmetics have no significant impact on the climatological mean state, 448 histograms of prognostic variables are also not changed ( Fig. 7a and b). However, the 449 tendencies are increased by orders of magnitude with 16-bit arithmetics (Fig. 7d and e), 450 as rounding errors cause gravity waves to radiate away from eddies (Fig. 7f). Gravity Tendencies are about 4 orders of magnitude smaller than the prognostic variables.

459
This poses a problem for number formats with a machine epsilon, measured as decimal 460 precision, significantly lower than 4 decimal places (Table 1). Float16 has a machine ep-461 silon of 3.7, which is presumably close to the lower limit beyond which the addition of 462 tendencies will be round back. The BFloat16 number format has a machine epsilon of 463 2.8, which explains why flow is stalling when simulated with BFloat16. ageostrophic velocities increase in variance, especially for BFloat16 (Fig. 6). Gravity waves 487 are similarly present for mixed precision although weaker for tendencies computed with 488 Float16 (Fig. 7d) and, as discussed, they tend to not interact with the geostrophic time 489 and length scales. Although the results show that Float16 is generally a preferable num-490 ber format over BFloat16 for the applications presented here, we acknowledge that the 491 conversion between Float32 and Float16 will come with some computational cost. In con-  BFloat16 introduces a larger error than for the other 16-bit formats as the decimal pre-516 cision is with 2.8 clearly lower (Table 1) for the range of values occurring within the prog-517 nostic variables ( Fig. 7a and b). -25-manuscript submitted to Journal of Advances in Modeling Earth Systems η as before and are up to about two orders of magnitude smaller than the errors that 519 result from 16-bit arithmetic. As even the worst 16-bit communication format, BFloat16, 520 has a smaller error than the best mixed precision formats, Float16 with Float32, we ex-521 tend the short-term forecast experiments to include two 8-bit formats, Posit(8,0) and Float8 522 (see Table 1

552
A perfect model is used in this study, such that any form of model or initial con-553 dition error is ignored and only the number format is changed between simulations. Solely 554 discretisation errors are estimated by lowering the spatial resolution by a factor of 2. Al-555 though this is essential here to analyse the impact of rounding errors isolated from other 556 errors, it is in general not a realistic configuration for weather or climate models. More 557 complex models include many other sources of forecast error, such that the contribution 558 of rounding errors from 16-bit arithmetic would likely be dwarfed by model, discretisa-559 tion or initial condition errors.

560
Only the most common discretisation method for fluid dynamics was used in this  Horizon 2020 research and innovation program under grant agreement 823988.

586
Link will be converted to DOI upon acceptance. Data can be reproduced from https:// 587 github.com/milankl/ClimateModels16bit and will also be made available in a repos-588 itory upon acceptance.