Wednesday, January 28, 2009

Deeper Look at the Apply Method in Scala

In Scala, there is a language feature generally referred to as "The Apply Method" that has the following rules:


  1. Any object that has an apply method can be called with the .apply omitted.
  2. Functions are no more than objects.


Let's take a look at an example. Given the following abbreviated definition of class Array, and an instance a,

class Array{
def get(index:Int) = { ...some code to get from the array... }
def apply(index:Int) = get(index)
}

val a = new Array(whatever)

Then the following calls are essentially equivalent:

a.apply(7)
a(7)
and also:

a.get(7)

(the call to get is only equivalent because apply simply calls get. apply could have any implementation, of course.)

While apply is really useful in cleaning up syntax, the real beauty of it is hidden just below the surface. (Note: Some of the following might not be implemented exactly as I describe, I'm not 100% sure, but for the sake of understanding, I think it's okay.)

Scala'a apply is a concept that can be considered universally across functions or methods, objects, anonymous functions, case classes, and the like. In Scala, a method is nothing more than a Function object containing an apply method which takes the same arguments as the method itself.

Let's take a look at a more in depth example. All three of the following are basically equivalent:
  
def shout1( word: String ) = println(word + "!")

val shout2 = new Function1[String, Unit] {
def apply(word: String) = println(word + "!")
}

val shout3 = (word:String) => println(word + "!")

All three are of type Function1[String,Unit], which has an apply method that takes a String and returns Unit. Notice that shout3 uses anonymous function syntax. This is really nothing more than syntactic sugar for Function1. All three can be called like so:

shout1("hey")
shout2("hey")
shout3("hey")

The following client code further demonstrates that they all derive from the same type.

def ohYeah( f: String => Unit ){
f.apply("oh yeah")
f apply "oh yeah"
f("oh yeah")
}

ohYeah(shout1)
ohYeah(shout2)
ohYeah(shout3)

However, one thing I find odd that cannot be done is the following:

shout1.apply("hey") // compiler error!
shout2.apply("hey") // ok
shout3.apply("hey") // ok

If anyone smarter than me cares to chime in on that one, I'd be quite happy. Apparently Scala doesn't always treat actual methods exactly the same as the equivalent Function values, but I'm not sure why. I tend to think it should. There must be a good reason.

The first rule applies to top level objects, functions, and objects defining apply, but what about classes? Are classes no more than objects in the language, the same way that functions are?

Unfortunately, it appears the answer is No.

Before we get into that, let's that a quick look at the exception to that - case classes. For any case class C, you can say C(), and that returns you an instance of that class. Hmm... C(). That looks an awful lot like what we've been looking at. As it turns out, it is.

In Scala, when you define a case class, a corresponding top level object with the same name is created. This object is a factory which creates objects of the original class. It contains an apply method that takes the same arguments as the constructor of the original class and simply calls that constructor.

A quick example should help:

case class Dood(name: String)

class DoodClient {
val a = new Dood("Jack")
val b = Dood("Daniels")
val c = Dood.apply("Cough")
}

The expressions in b and c are equivalent. b is just shorthand for c. In both lines, Dood refers to the top level object Dood, and not the class.

Knowing all this, I should correct myself. I said case classes are an exception to the rule that classes aren't simply objects in the language. This isn't really true. Creating the case class Dood creates both a class and a top level object. It's that object that has the apply, and the class itself still isn't a full fledged object.

But why? Well, I think the short answer is that on the JVM classes aren't top level objects. That seems like a cop out answer though. Let me explore it for a bit, though I'm really just thinking out loud.

If they were top level objects containing an apply method then we could probably do away with the new keyword entirely, and then any call to any object would simply be a call to apply. Things would be entirely consistent both from the internal language representation, and from the perspective of the developer as well.

But what are the consequences of this? I guess I'm not sure. I'm going to have to think through it a bit harder. I'd appreciate comments from anyone with more knowledge on the subject than myself.

10 comments:

  1. You have to "tie the knot" somehow. If a method was just sugar for an instance of Function, then Function's apply would be a Function, but Function defines apply, which is a Function, which...

    In Scala methods are "primitive" in the sense that they are not in fact objects. But Scala does have a way to promote them to objects.

    def shout1( word: String ) = println(word + "!")
    shout1: (String)Unit

    val f = shout1 _
    f: (String) => Unit = [function]

    f.apply("hello")
    hello!

    f("good bye")
    good bye!

    There are other important differences between methods and Functions. A method can be "generic"

    def id[T](x : T) = x

    But a Function cannot. Other languages like SML and Haskell '98 have a similar restriction, which they call the "monomorphism restriction."

    As for why new isn't always just a function call on a meta-class - I'm not sure. There are probably limitations due to Java compatibility that require Java-ish constructors. Constructors have rules that say that super constructors will be called before child constructors are called, etc. Factory "apply" methods don't have that kind of restriction.

    ReplyDelete
  2. (shout1 _).apply("hey") will do what you want. You can always turn a method into a function by appending the underscore.

    ReplyDelete
  3. Oh good, make a little dinner before clicking send and james iry will beat you in for sure. He's the scala blog terminator.

    ReplyDelete
  4. Guys, Thanks.

    I already knew about the _, and I've used it, mostly for method aliasing. But, I suppose I still don't understand exactly how it works, and certainly don't understand why they chose that syntax.

    I guess my main question was simply - why aren't methods treated as first class objects in all situations. The fact that I can say ohYeah(shout1) certainly leads me to believe that in some situations they are treated like Functions. How does that work? Is there an implicit conversion somewhere for cases like that? Or really just something built into the compiler?

    James, you somewhat answered why, with the ad infinitum recursion. However, I think its possible to get around that problem by treating only apply methods specially, that way all other methods could be treated simply as objects.

    I still think a deeper explanation is in order.

    And I suppose I better read up on the monomorphism restriction.

    Thanks.

    ReplyDelete
  5. Maybe methods are only promoted to objects when used as an argument.

    ReplyDelete
  6. Thanks.

    I should probably update the post, as it suggests methods are first class objects. I believe they should be. I think it can be done better. Maybe I'm crazy. I'll try to write up some wacky ideas on how it might work. Slight problem is, I'm somewhere between the working world and the academic world. Not good enough to be an academic, not happy enough writing web sites. So, I think all my ideas come off as wacky to everyone! Hehe.

    ReplyDelete
  7. Great article Jack! I'm going to link it from my own Scala article on my blog if you don't mind.

    ReplyDelete
  8. The article could have mentioned:

    class A { def apply = 5 };
    (new A)()

    To show that it's a consistent Scala feature.

    ReplyDelete
  9. @Michael, of course. (Sorry I'm a year and a half late answering)

    @Pedro, Thanks. It's not half bad for 3.5 years ago though, and I'm definitely too lazy to update it. That's what comments are for :)

    ReplyDelete