Raising a number to a non-integer power.

[SiliconWizard]

Actually, the form a^b = 2^{b log₂ a} is useful, as it is something a computer can do in binary directly, and quite efficiently for base-2 floating point numbers. Details.

Yep!
Generally speaking, even for integer exponentiation, a binary algorithm is much more efficient than a simple linear algorithm (multiplying the number by itself b times.)

[Nominal Animal]

For those interested, the binary integer algorithm calculates a^b for integer b>0 with at most 2 log₂ b multiplications. In pseudocode,

Code: [Select]

Function ipow(a, b):
    Let result = 1
    While b > 0:
        If b is odd:
            result = result × a
            b = (b - 1) / 2
        Else
            b = b / 2
        End If
        a = a × a
    End While
    Return result
End Function
For negative b, you do 1/ipow(a,-b) or ipow(1/a, -b).  b=0 is mathematically ambiguous case, and usually handled separately.
Programming languages with integer division (dropping the fractional part, or rounding toward zero, or binary shift right by one bit), just do b = b / 2 in both branches of the If clause.
The one extra squaring of a is avoided in practice by moving the end-of-loop test to just after halving b.

[RoGeorge]

Op asks what is the intuitive representation of raising a number to a fractional power, gets answers about best numerical algorithms.   

Software devs are the best example for the saying "When you have a hammer, everything looks like a nail".  Not sure if this is their biggest flaw or their biggest power.  And, if that's a power indeed, is it a non-integer one?   

[magic]

Software devs are the best example for the saying "When you have a hammer, everything looks like a nail".  Not sure if this is their biggest flaw or their biggest power.

Look at the software available today, compare with the software available 15 years ago, take a wild guess

[aneevuser]

Fair enough, off the top of my head I can't provide a simple algebraic argument why complex exponents should be defined the way they are. That doesn't necessarily mean no such argument exists. I don't know what reasoning originally led to the Euler identity and things like that - perhaps the Taylor series, maybe something more direct and straightforward.

You might want to take a look at the first chapter (and others) of "Visual Complex Analysis" by Needham, which gives a couple of suggestive geometric/algebraic arguments for the validity of Euler's formula.

[Nominal Animal]

Op asks what is the intuitive representation of raising a number to a fractional power, gets answers about best numerical algorithms.   

Hey!  It was a clarification to a comment why expressing a fractional power a^b using base-two exponent and base-two logarithm, 2^{b log₂ a} is a practical way to look at it, because that's how computers deal with it.

Let's recap:

One way to look at non-integer powers is to look at fractional powers:
    a^b/c = a^b · a^1/c
The second term is the same as c'th root of a, by definition.  So, for example 2^3.5 = 2³·2^0.5 = 2³·√2 = 8√2.

This approach has the problem that it only works for rational numbers – those that you can express as the ratio of two integers –, you cannot calculate e.g. 2^π with it.  Besides, it's slow and burdensome.

To solve this problem, John Napier introduced logarithms in 1614.  The basic equivalence with exponentiation is
    e^x = y  ⇔  x = log_e y
It's main purpose is to make multiplication and division easier.  Logarithm turns multiplication into addition, division into subtraction, using just standard multiplication rules (for powers), with exponentiation as its inverse:
    A · B = e^{log_e A} · e^{log_e B} = e^{(log_e A) + (log_e B)}
    A / B = A · B^-1 = e^{log_e A} · e^{- log_e B} = e^{(log_e A) - (log_e B)}

Technically, logarithms are defined only for nonnegative reals, but we can treat the signs of A and B separately anyway in multiplication and division trivially: if we have an odd number of negative values, the result is also negative; otherwise it is positive.  Easy peasy.

Mathematically, this equivalence gives us a simple way to express any power via exponentiation and logarithms as
    a^b = e^{b log_ea}

Because e is an irrational number itself (it has infinite number of digits in any integer base, be it decimal or binary or other), it is something that computers cannot handle directly, and can bother us humans quite a bit, too.  So, instead, the base 2 one is used in computers (and some humans, too):
    a^b = 2^{b log₂a}
There is lots of information about the base-2 logarithm, or binary logarithm, for example at Wikipedia.

But wait, didn't we just come around a full circle, and just answer the question with "by raising 2 to a fractional power you calculate thus"?

No, because raising 2 to a fractional power is a special function we can call base-2 exponential.
You are probably already aware of the exponential function, exp(x) = e^x.
The base-2 exponential is very similar, except there are algorithms how a computer (or a human counting in binary) can approximate it directly, to whatever precision you like.  So, while it looks like the same problem, technically the procedure is
    a^b = exp2(b log₂ a)

How one should grasp it intuitively, then?

Think of integers in the number line.  They're distinct, with a gap in between.  We use real numbers to describe values that do not match any specific integer, but fall in between two integers.  Positive integers can count things, so they're very intuitive, even many animals can do it.  Reals are more difficult.  Rational numbers are ratios of two integers, but they aren't all reals; we also have irrationals like π that cannot be expressed exactly as a ratio of two integers.  So, it is better to think of real numbers as being in between integers, without a perfect real-world analog.

In the exact same way as real numbers extend integer points to a continuous line, non-integer powers extend integer powers to a continuous curve.  Non-integer powers behave exactly like integer powers, mathematically.  So, the best way to intuitively grasp it, is an extension to the integer powers, really.

So yeah, we really don't have a good analog for non-integer powers.  One tripping point is that if we want to calculate the numerical value without having a tool that can do it directly for us, we'll need exponentiation and logarithms to do so.  You might think that is because they are somehow special, but it is quite the opposite: exponentiation and logarithms are just the tool we use for the extension, and themselves have a very clear, intuitive definition.

Is this any better, RoGeorge?  I bet it is five times more than most members here care to read, so that's why I tried to only drop in the minimal nugget.  Would've been much better as a video, too.

[RoGeorge]

In the exact same way as real numbers extend integer points to a continuous line, non-integer powers extend integer powers to a continuous curve.

Perfect, hard to explain in less words!   

The next chart first represents 4¹, 4², 4³, 4⁴, 4⁵ as red dots.  Looks like the red dots nicely align like the beads on an invisible string.  That invisible string is our function 4^x, drawn as a green dotted curve.  If we stick a blue dot on that green curve, and move it left and right, that would show values for fractional power, and if we drag it to x = 3.9, the blue dot will show us the value for 4^3.9.

It will be almost like 4 multiplied 4 times, but a little less than 4x4x4x4.



For a live, interactive chart
https://www.desmos.com/calculator/s9zbhc9mcy

[Nominal Animal]

Exactly.

Visual examination also gives a good starting point for intuition on the behaviour of the exponential function and the logarithm.

The curve y = exp(x) has slope x at x=x, and the natural logarithm curve, y = log(x), has slope 1/x at x=x.

This kind of extension is needed for us humans, because our intuition is not sufficient to handle all the things we can understand.  We need logic, and mathematical tools we don't have intuitive real-world analogs for, only definitions.

As an example, consider functions.  There is something called a Lambert W function, which cannot be expressed in terms of elementary functions at all; we cannot express it exactly in form W(z) = ... .  We can approximate it numerically, though, to any desired precision you want.  Anyway, the Lambert W function is defined as the function that satisfies W(z) exp(W(z)) = z.  This means that for example, if you wanted to solve exp(-z) = z, the answer is z = W(1) ≃ 0.56714329.  Or, if you wanted to solve exp(-z)=z / 2, the answer is z = W(2) ≃ 0.8526102.  Except that W(1) and W(2) are the exact answers, and when looking for a behaviour of something ("What kind of function fulfills these rules?"), the answer just might be W(z).

Before you think that that the Lambert W function is just a mathematical curiosity, I'll just point out that it is the function that provides the exact solution for the double-well Dirac delta function model for equal charges: for example, the hydrogen molecule.  It's not a curiosity or "pure math", it is an useful tool for describing stuff in nature.  It's just hard to pin down in "human terms".  At some point, you just have to accept that some of the mathematical tools you use don't have good real-world analogs or plain explanations, and that knowing their operating context and rules is enough – unless you're a mathematician.  I ain't.

[RoGeorge]

I'm not familiar with the Lambert W function, but calling that a function feels like cheating.  By definition, a function can not have more than one output for a single input.

Maybe in math we can change or extend the definition for what a function is, but then it will be out of this world.

In the physical world, when for a single input we observe multiple results, it means we are missing something, like an extra input variable we disregarded, or an extra function/phenomenon we don't realize it's there.

[T3sl4co1l]

It's a function if you specify a main branch, right.  That can sometimes be hand-wavy when it shouldn't be.

Just as ln(z) is not a function, it has infinite solutions ln(z) = ln(z) + 2n pi i; by convention we take the main branch (-pi < Im(ln(z)) <= pi), Ln(z).

Tim

[SiliconWizard]

I'm not familiar with the Lambert W function, but calling that a function feels like cheating.  By definition, a function can not have more than one output for a single input.

Maybe in math we can change or extend the definition for what a function is, but then it will be out of this world.

This is called a multivalued function, and this isn't an ordinary function indeed.

https://en.wikipedia.org/wiki/Multivalued_function

In the physical world, when for a single input we observe multiple results, it means we are missing something, like an extra input variable we disregarded, or an extra function/phenomenon we don't realize it's there.

Quoting Wikipedia:

In physics, multivalued functions play an increasingly important role. They form the mathematical basis for Dirac's magnetic monopoles, for the theory of defects in crystals and the resulting plasticity of materials, for vortices in superfluids and superconductors, and for phase transitions in these systems, for instance melting and quark confinement. They are the origin of gauge field structures in many branches of physics.

[Nominal Animal]

Yup!

I used Lambert W as an example of a "completely unintuitive function-like thing" that is very useful but almost impossible to find real world analogs or useful intuitive reasoning for, to show why we must not expect to have good analogs or intuitive explanations for all our mathematical tools.

When I first started learning calculus, I had a hard time giving up wondering "what value" δ(0) is.  δ(x) is the Dirac delta function.  δ(x)=0 for all x except x=0, but its integral over all real x is 1.  I internally settled for something like "infinitesimal infinity" (I don't think I've ever said it aloud), before I realized the entire question is nonsense.  δ(x) is a mathematical tool with specific rules and behaviour, and does not necessarily have that property at all!  Indeed, if you find that to solve a problem you need the value of δ(0) outside any differential equation or similar context, you can be sure that you went wrong somewhere.

Also consider dimensional analysis.  In dimensional analysis, you drop all quantities (numbers) and apply your model (equation or formulae) to the units only, and see what kind of unit pops out.  If you get e.g. m/s where you expect time units, you know that either you made a mistake, or the model is b0rked.  But working out a formula or equation with units, and completely without numbers or variables, is pretty counter-intuitive to most people.  Many learn to just ignore the units and work with the numbers only, and that sometimes leads to really bad mistakes.

[T3sl4co1l]

I used Lambert W as an example of a "completely unintuitive function-like thing" that is very useful but almost impossible to find real world analogs or useful intuitive reasoning for, to show why we must not expect to have good analogs or intuitive explanations for all our mathematical tools.

Another way to express it could be as returning a vector, i.e., a function \$ \mathbb{R} \rightarrow \mathbb{R}^n \$, and more generally than that, functions of arbitrary dimension in the domain and range.  This requires more formalism to pack up (i.e., are we using linear algebra, set theory, etc.?), but the basic idea can be seen.  Applied in another domain, we might think of functions in computer science as some dimension in the parameters, and the return values (assuming pure functions, i.e. no side effects, no internal state).  We introduce type theory (the vectors need to be not just the same number of elements, but the same types as well).  And then we can think about data and its typing in various useful ways.

When I first started learning calculus, I had a hard time giving up wondering "what value" δ(0) is.  δ(x) is the Dirac delta function.  δ(x)=0 for all x except x=0, but its integral over all real x is 1.  I internally settled for something like "infinitesimal infinity" (I don't think I've ever said it aloud), before I realized the entire question is nonsense.  δ(x) is a mathematical tool with specific rules and behaviour, and does not necessarily have that property at all!  Indeed, if you find that to solve a problem you need the value of δ(0) outside any differential equation or similar context, you can be sure that you went wrong somewhere.

Indeed, when a continuous function is desired, a common implementation is a Gaussian with lim width-->0 and height-->inf, taking the joint limit such that the integral property is preserved.  This gives a rigorous basis and allows some analyses which would otherwise fail on the more basic definition.  But it is indeed a not-function, in its basic form, and I think most instructors are careful to point that out when it's introduced (at least, mine were).

Also consider dimensional analysis.  In dimensional analysis, you drop all quantities (numbers) and apply your model (equation or formulae) to the units only, and see what kind of unit pops out.  If you get e.g. m/s where you expect time units, you know that either you made a mistake, or the model is b0rked.  But working out a formula or equation with units, and completely without numbers or variables, is pretty counter-intuitive to most people.  Many learn to just ignore the units and work with the numbers only, and that sometimes leads to really bad mistakes.

Yup, and since dimensions are conserved under arithmetic operations, you can assign dummy dimensions to mathematical variables (which might otherwise be dimensionless), and track down some algebraic errors in the process (when you're working large equations by hand).

And it's not just a physicist's hand-waving tool, it has rigorous application!  Using a change of variables, units can be extracted from otherwise very difficult expressions.  Feynman integration is a famous case, where the units are extracted from an integral, so that the integral is over an abstract mathematical function, which then merely returns a dimensionless geometric constant.  All the dimensional relationships are basic arithmetic, so that qualitatively speaking, we have everything we need to know about the problem, and exact results are merely proportional to that.

And we can then compute that function through any experiment or dynamical system which is equivalent to it, not just the problem in question.

Tim

[TimFox]

The rigorous definition of the Dirac delta is not a function, but a “distribution”, sometimes called a “generalized function”.  Although the original concept came before him, it was developed and tidied up by Laurent Schwartz midway through the last century.  See Wikipedia for a summary.  There is no difference between his formulation and the results obtained with common concepts involving shrinking tall Gaussian curves to the limit, but it is a good example of mathematics catching up with a useful bit of physics.
(In my youth, we studied the history of the Einstein Summation Convention that was held discretely on the Kronecker Delta.)

[Nominal Animal]

The rigorous definition of the Dirac delta is not a function, but a “distribution”, sometimes called a “generalized function”.

In the context of this thread, "generalized function" would be key    (Because the mathematical concept of "distribution" doesn't really match what a layman might think a distribution is, you see, and this thread is all about our human need for intuitive real-life analogs.)

From generalized functions or "distributions" (in the mathematical, not traditional statistical sense) one gets to functionals and vector spaces and linear algebra and whatnot, all very useful tools.

Me, I was never particularly good at math, and what little I know, is all hard-fought.  I didn't really grok differential calculus until I started working with interatomic potentials.  Before, I could solve problems I recognized, sure; but it was just applying methods I knew, like treating each problem as a perp and looking them up in my mental records, then applying the suggested methods therein.  No true insight, no creativity, just straightforward grunt work.  Now, it feels different.  Enjoyable.  Like a very nice, well-worn, reliable tool.

Or I might just be going bonkers in my middle age, and sliding waaay back on the D-K scale!

[TimFox]

English terms for mathematical concepts are usually existing words, already in use in common language.
"Function" (v) can mean to work or operate in a desired way
"Function" (n) can refer to a formal social event.
The advantage of using "standard" nomenclature is the ease of locating information, as in the Wikipedia article on "distribution".  I like Wikipedia's term "disambiguation" for separating out different meanings of the same word.

[Nominal Animal]

I wasn't disagreeing, just trying to make sure others reading this thread won't accidentally conflate it with statistics, TimFox.

In this kind of a thread, where we're looking at ways to help others understand stuff better, I've found that carefully picking the terms to help steer those climbing the ladder helps, even if the terms used aren't the conventional or usual ones; the implicit associations matter.

I do agree that distribution is the better term, in general.  But, in the context of this particular thread, the underlying key is extension or generalization of mathematical tools we have real world analogs for to those we have not.

As an example, when talking about unit quaternions in the context of spatial rotations, I like to use the term versor, because many (programmers at least) have encountered quaternions and found them untractable or unreasonably hard.  The truth is, unit quaternions are easy, much easier than Euler or Tait-Bryan angles, and perfectly useful for anyone working in 3D on computers or microcontrollers.  In the realm of spatial rotations, versors really only use a couple of details from quaternions proper (Hamilton product, really).  Slap on conversion to a 3D rotation matrix (of which there is only one, unlike Euler or Tait-Bryan angle conventions), addition/interpolation properties (and how renormalization to unit quaternion does not have any directional bias, unlike E/T-B), plus possibly recovering the versor from a pure 3D rotation matrix, and it's all there.  It all fits one one or two sheets of paper.  Picking the name "versor" is just a psychological tool to avoid the association with any preconceptions on how quaternions are "hard".  Since they haven't any preconceptions against "versor", they tend to be very surprised how simple the operations and their implementation are.

[basinstreetdesign]

What is this number i that you speak of?

You must be 'j'oking

Tim

He probably just imagined it.

Get real!

[RoGeorge]

What is this number i that you speak of?

You must be 'j'oking

Tim

He probably just imagined it.

Get real!

That's not natural!   

[Brumby]

What is this number i that you speak of?

You must be 'j'oking

Tim

He probably just imagined it.

Get real!

That's not natural!   

 

[Brumby]

Also consider dimensional analysis.  In dimensional analysis, you drop all quantities (numbers) and apply your model (equation or formulae) to the units only, and see what kind of unit pops out.

Back in high school, I did this when looking at E=mc² to find the units needed to match with E and m was a velocity.  Now I know this actually has a name (many decades later).  (Why it was the particular numeric value of the speed of light was a further question - but one which I did not pursue.)

Yup, and since dimensions are conserved under arithmetic operations, you can assign dummy dimensions to mathematical variables (which might otherwise be dimensionless), and track down some algebraic errors in the process (when you're working large equations by hand).

That's a neat trick.  I like it.

[magic]

In the exact same way as real numbers extend integer points to a continuous line, non-integer powers extend integer powers to a continuous curve.  Non-integer powers behave exactly like integer powers, mathematically.  So, the best way to intuitively grasp it, is an extension to the integer powers, really.

OK, but there is infinitely many such extensions (even continuous and infinitely differentiable) that differ in the exact curvature between your chosen points. You really need additional arguments to demand this one particular extension that's currently accepted.

So yeah, we really don't have a good analog for non-integer powers.  One tripping point is that if we want to calculate the numerical value without having a tool that can do it directly for us, we'll need exponentiation and logarithms to do so.  You might think that is because they are somehow special, but it is quite the opposite: exponentiation and logarithms are just the tool we use for the extension, and themselves have a very clear, intuitive definition.

I would agree it's clear and intuitive what 2+2 is, but not sure what's intuitive about the definition of ln(sqrt(2))

Moreover, there is absolutely no readily apparent reason why a^b defined in terms of exp/ln should give the results we expect when a and b are natural.
Unless you define exp in terms of e^x, but you can't do that because you just defined a^b in terms of exp

[Someone]

a^b makes a perfectly logical extension to real values of b. x! for positive reals, still plausible; but over the whole reals.... thats mind bending.

[T3sl4co1l]

a^b makes a perfectly logical extension to real values of b. x! for positive reals, still plausible; but over the whole reals.... thats mind bending.

Well, literally impossible, too; a typical analytic extension of factorial has a pole at (-1)!, so further extension isn't meaningful.  However, if we integrate around the poles taking the principal value, we get a meaningful answer again (with finitely many exceptions: the poles themselves).  Thus it works for complex numbers, and you can find real (non-integer) values after all.

Tim

[Nominal Animal]

In the exact same way as real numbers extend integer points to a continuous line, non-integer powers extend integer powers to a continuous curve.  Non-integer powers behave exactly like integer powers, mathematically.  So, the best way to intuitively grasp it, is an extension to the integer powers, really.

OK, but there is infinitely many such extensions (even continuous and infinitely differentiable) that differ in the exact curvature between your chosen points. You really need additional arguments to demand this one particular extension that's currently accepted.

No, there is only one the fulfills the standard arithmetic rules of addition and subtraction in the exponent for both integers and non-integers (reals).  I mean, for which exp(a+b) = exp(a)·exp(b) and exp(a-b) = exp(a)/exp(b).

So yeah, we really don't have a good analog for non-integer powers.  One tripping point is that if we want to calculate the numerical value without having a tool that can do it directly for us, we'll need exponentiation and logarithms to do so.  You might think that is because they are somehow special, but it is quite the opposite: exponentiation and logarithms are just the tool we use for the extension, and themselves have a very clear, intuitive definition.

I would agree it's clear and intuitive what 2+2 is, but not sure what's intuitive about the definition of ln(sqrt(2))

I meant that after all the math work to prove their properties are what has been claimed, a layman can grasp exponentiation and logarithm in very simple terms:
exp(x) is the curve that has slope x at x, and has value exp(0)=1.
log_e(x) is the curve that has slope 1/x at x, and has value log_e(1)=0.

You do need lots more math to prove this is so, or to discover why this is so, but for an intuitive understanding of what they are (as opposed to why or how), this suffices.

Unless you define exp in terms of e^x, but you can't do that because you just defined a^b in terms of exp

Let's not try and be cute here. exp(x) ≝ e^x. I use the form exp(x) to remind the reader that this is not a cyclical definition; that we can define e^x for all real x as a function that does not rely on being able to raise a value to a non-integer power.

It is well and good (and I definitely appreciate it) if you(or anyone else) point out errors or holes in my argument, but in the context of this thread, it is not worthwhile to point out the lack of mathematical rigor, because the issue here is to not prove or claim anything, but to offer A) an intuitive grasp of non-integer powers and B) to show why the need of such intuitive grasp is hindering our use of math as a tool, and why letting go of it, instead understanding that only some of math has a real-world analog or an intuitive explanation and the vast majority is an extension without, is useful and productive.


So, to recap the A) point, in the hopes to clear any confusion, because me fail English often:

We can define non-integer powers like a^b = exp(b·log_e(a)).  Indeed, computers typically calculate arbitrary real powers a^b = 2^{b·C·log₂ a}, where C is a constant (C = log_e 2).

While exp(x) is actually e^x, where e is the number known as Euler's constant (approximately 2.71828), this is not a cyclical argument, because we can define, and understand, exp(x) and log_e(x) differently, as functions that we can define without relying of non-integer powers at all.

Indeed, exp(x) is the curve that has slope x at x, and has value 1 at exp(0).
Similarly, log_e(x) is the curve that has slope 1/x at x, and has value 0 at log_e(1).

(Exactly why and how this is so, requires more math, and goes beyond the scope here.)

For example, if you needed to calculate exp(x) for some x by hand, you could do so by using the series definition, exp(x) = 1 + sum x^k / k! for k = 1, 2, 3, ... and so on, where k! is the factorial of k (product of all integers from 1 to k, inclusive).

While we do have some series we can use to calculate log_e(x), they are too slow (need too many terms) to be practical.  Instead, we used to use tables of logarithms and exponentials instead, y = exp(x) ⇔ x = log_e(y) (listing both y and x), based on pre-calculated exponentials. However, log₂(x) = log_e(x) / C, is much easier to calculate.  Indeed, computers typically calculate log_e(x) = C·log₂(a) instead.

(For the exact description of how to calculate log₂(x), see e.g. the Wikipedia Binary logarithm article.  Most computers approximate real numbers with floating point numbers, where x = m·2^b, where m is the mantissa (and normally represents a value between 0.5 and 1.0) and b is the exponent.  The binary logarithm is especially powerful for numbers expressed in this form.  For example, since 1987 or so, the Intel x86 (and x86-64 or AMD64) processor families have had it in hardware: a machine instruction FYL2X that calculates the binary logarithm of a given value, and another, FYL2XP1, that calculates the binary logarithm of one plus the given value, for even more accurate results near 1.)

To recap point B), most mathematical tools do not really have real-world analogs.  We can use integers for counting, and rationals for ratios, but what about irrationals: real numbers that cannot be represented exactly by any ratio?  We know π, and it is irrational, so they are definitely useful.  Earlier in this thread, I mentioned the Lambert W function, which isn't really a function because we cannot write it in form W(x)=something at all, and belongs to a group called generalized functions, or more properly, distributions (but don't confuse them with statistical distributions; they have some similar properties but are a different thing in math).  Many tools in math can be considered generalizations of the things that do have real-world analogs, of the things we can grasp intuitively, but cannot themselves be grasped intuitively (like irrational numbers).

I myself have never been good at math.  I might say I'm good at applying the math I know to solve problems, but there are lots and lots of mathematicians that can actually create new mathematical tools!  I believe that having the need for real-world analogs was hindering me for years, until I understood and accepted that not everything has such, because they are more like extensions; like using a waldo arm instead of your own arm to manipulate hazardous materials, except for your mind.