Initial/Inlet turbulence is not easy to define. But you also want to pay attention to the sensitivity of your results to these inlet values. Some models are more sensitive than others in a number of flow fields.
There are a number of ways to define inlet turbulence, some involving the use of free stream Mach number and speed of sound etc. You can refer to NASA’s turbulence modelling resource for more information.
The equations listed above are for fully developed internal incompressible flow. To use an omega-based model in such case, you want to provide (inlet turbulent intensity) I and an (estimated length scale) L. The value I and (turbulent kinetic energy) k are calculated by the two equations you wrote down. To calculate (turbulent dissipation rate) epsilon, use the mixing length formulation, because OpenFOAM uses mixing length:
. This also means L is 0.07*(hydraulic diameter). Now that you have epsilon, you can go ahead and calculate (turbulence frequency) omega, again using the mixing-length formulation.
Note that this way the (turbulent viscosity ratio) nut/nu at inlet will be very large, which is reasonable for fully developed internal flow.
It is more difficult to define inlet turbulence for external flows, because now the length scale can be less obvious, and sometimes you might need to define a very large omega at inlet so that nut/nut is small — for a clean inflow with negligible turbulence, the ratio nut/nu should be a small value, meaning the laminar viscosity is dominant. Such flow fields are typical when computing external flow with a fairly large domain ( not in a wind tunnel ).
In terms of the instability of your model, omega-based models tend to create less turbulent viscosity than standard k-epsilon around the wall. This is one of the reasons why omega-based models can produce more flow instability in the wake for external flow, and as a result, forces fluctuate a lot. If you increase inlet I and/or nut/nu, the omega-based model will have a more stable wake. If you can’t introduce more turbulence at inlet, you can take the time-average of those force values or switch to a different turbulence model.