Filed under: scala

Go vs Scala vs NodeJS

These test are run with:

ab -c 100 -n 50000 http://localhost:8000/

The memory usage is analyzed with : pmap $PID | tail -1 >> mem.txt; SLEEP 0.001

  Req/s Mem  
Go 9428.90 ~18.5M  
NodeJS 5775.84 ~646M  
Java(Netty) 7086.34 ~2 434M (No flags)
" 6931.62 ~512M (-server -Xmx128m -XX:+UseConcMarkSweepGC)
" 9682.42 ~577M (after a couple of runs for JIT warmup )

Conclusion ?

The mem test takes into account all the libraries loaded by the process as this seemed to be the fairest comparison.

The Java version might be doing a bit more than the the others since it was an experience I had here... Without explicit JVM flags it gets very memory hungry and I bet that a -server flag without memory restrictions would be a huge mem hog. Fortunately the last tests show that with a memory usage similar to NodeJS's it does a whole lot better (I'm happy cause I can keep using Scala instead of Javascript :P)

NodeJS has a lot of hype around it and it definetly deserves it but without the web workers support it is not capable of taking full advantage of several cores and you'd have to go with a local pool and a load balancer in front of it. It's memory usage is not the best.

Finally Google's Go is surely the big winner here. With very little memory usage and good performance it is the undisputed champion. Also, I'm using the 6g compiler but have read somewhere that gccgo could bring a 50% improvement. The language's not great but it sure beats C, C++ and even Java in my book ;)

Scala on Android 101 - Proguard, XmlParser and Function2AsyncTask

I've neglected my blog for a while so I though I'd share my latest experiments with running Scala on Android. In order to try a couple of things with Scala on Android I'm creating an Activity which  basically trying to consume a WebService and display the results in a List.Please keep in mind that I'm only learning Scala now so I'll gladly get any comments and/or suggestions regarding my solutions.These are the problems I've found  :

  • PROBLEM #1
    • The scala-library.jar is too big to dex so you have to use ProGuard to keep things small. Try making it work properly with maven and you'll have issues with getting all the plugins working in the right order!
  • SOLUTION #1
    • Ditch maven! "But having it manage my dependencies is so cool..." I hear you say.. well let's use Ivy:

Just use one of the numerous ANT build scripts for Android and add the targets for Ivy and ProGuard and the scala repository to your ivysettings.xml:

.....
<url name="scala-tools.org">
     <artifact pattern="http://scala-tools.org/repo-releases/[organisation]/[module]/[revision]/[module]-[revision].[ext]" />
  </url>
....

add your dependencies to your ivy.xml:...

<dependency org="org/scala-lang" name="scala-library" rev="2.8.0.r18462-b20090811081019"/>
<dependency org="org/scala-lang" name="scala-compiler" rev="2.8.0.r18462-b20090811081019"/>
<dependency org="net/sf/proguard" name="proguard" rev="4.3"/>

....

Add some nice Ivy targets:

<taskdef resource="org/apache/ivy/ant/antlib.xml"
     uri="antlib:org.apache.ivy.ant" classpath="${ivy.jar.dir}/ivy.jar"/>
 <target name="init-ivy" depends="download-ivy">
         <ivy:settings file="${basedir}/ivysettings.xml" />
        <ivy:retrieve />
</target>
  <target name="download-ivy">
    <mkdir dir="${ivy.jar.dir}"/>
    <get src="http://www.integratebutton.com/repo/
       ${ivy.install.version}/ivy-2.0.0-beta2.jar"
      dest="${ivy.jar.file}" usetimestamp="true"/>
  </target>

a nice ProGuard target:

    <target name="proguard" depends="compile">
      <taskdef resource="proguard/ant/task.properties"
               classpath="${external-libs}/proguard-4.3.jar" />
      <proguard>
-injars ${outdir}/classes:${external-libs}/scala-library-2.8.0.r18462-b20090811081019.jar(!META-INF/MANIFEST.MF,!library.properties)
-outjars ${outdir}/classes.min.jar
-libraryjars ${android-jar}
-dontwarn
-dontoptimize
-dontobfuscate
-dontskipnonpubliclibraryclasses
-dontskipnonpubliclibraryclassmembers
-allowaccessmodification
-keep public class <your main class here>
-keep interface scala.ScalaObject
      </proguard>
    </target>

and you're good to go!

  • PROBLEM #2
    • Scala uses the Java SAX parser which kept complaining about the use of namespace prefixes so I had to come up with a way to toggle this feature.
  • SOLUTION #2
    • Use my own SAX parser instance with the right features toggled :
object XmlParser {
  // Workaround for namespace prefix
  private val namespacePrefixes = "http://xml.org/sax/features/namespace-prefixes"
  val parser = javax.xml.parsers.SAXParserFactory.newInstance()
  parser.setNamespaceAware(false)
  parser.setFeature(namespacePrefixes, true)
  def load(i:InputStream) = XML.withSAXParser(parser.newSAXParser()).load(i)
}
  •  PROBLEM #3 :
    • AyncTask requires you to override an abstract method with varargs which is a no no right now in scala Ticket #1459
  • SOLUTION #3
    • Create a java class which extends AsyncTask, I called it MyAsyncTask:
import android.os.AsyncTask;
public abstract class MyAsyncTask<T1,T2,T3> extends AsyncTask<T1,T2,T2>{
        protected T2 doInBackground(T1 ...f) {
                return doInBackground(f[0]);
        }
        abstract protected T2 doInBackground(T1 f);
}

As you can see now doInBackground gets a single argument... which in my current implementation is a Function =)I've created an AsyncTask class in scala which extends MyAsyncTask and gets a Function as it's constructor arg :

class AsyncTask(f:()=>Unit) {
    def doInBackground {
        new _root_.pt.inevo.android.meo.MyAsyncTask[Function0[Unit],Void,Void]  {
           override protected def doInBackground(f: () => Unit):Void = {
               f()
               return null;
           }
          def onProgressUpdate(progress:Int*) { }
         override def  onPostExecute(result:Void) { }
   }.execute(f);         } }

I also added an implicit to convert from Function into my new AsyncTask class and wrapped it in an object:

object AsyncTask {
     implicit def function2AsyncTask(f: ()=>Unit):AsyncTask=new AsyncTask(f)
}

With this I can simply have regular functions and just call doInBackground:

val getChannelList = () => {
      for( c <- EPG.getChannelList \\ "Channel" ) {
         Log.v("getChannelList", c \ "Name" text )
}
override def onCreate(savedInstanceState: Bundle) {
      super.onCreate(savedInstanceState)
      setContentView(R.layout.main)
      getChannelList.doInBackground
}

I hope this helps some of you get started with Scala on Android. I'll be working on the layout stuff trying to get ListView and the Adapters to work properly.P.S: I'm using the great DataBinder Dispatch library for Scala to call the service through REST so my Service object is something like:

 object EPG {
    val req = :/("services.sapo.pt") / "EPG"
    def getChannelList=Http(req / "GetChannelList" >> as_xml)
}
 where the as_xml function is defined in my XmlParser class as:
def as_xml(res:InputStream) = XmlParser.load(res);