bank-vaultTo create a Web Service that allows for a third party to execute arbitrary code, might sound like a ridiculous, and dangerous idea. However, if you do, you can often reduce the number of HTTP requests your clients are doing to your Web Service by an order of magnitude, or even more! The problem is how you could allow for it, without exploding your system, and breaking every single security best practice we’ve ever conceived.

First you start out by only allowing for encrypted and signed Web Service invocations to execute. The mechanism you should probably choose here, is called PGP, and means Pretty Good Privacy. PGP allows you to be 100% certain about that the Web Service requests comes from a source you trust, in addition to encrypting all communication between your clients and your web server.

You’re still left with another problem though, which is that even though you trust the client invoking your Web Services, these clients might still be compromising your system. Either because they’ve been hacked, or because the developer creating your client system has done a sloppy job, and created a bug. Hence, you’ll need to make sure that no malicious code can possibly be executed by third party clients. The way you do this, is to semantically inspect the execution tree given, and check it against a “white list” of legal keywords and/or methods/Active Events. Luckily, pf.lambda has all of the mechanisms necessary to accomplish all of these things, almost with no hassle.

If you wish to replicate what I am doing in the above video, you’ll need to create a page in system42 called with the URL of “/execute-code”, and put the following code inside of it;

/*
 * making sure method is of type "POST"
 */
pf.web.request.method
pf.web.request.headers.get:Content-Type

if:@/-2?value
  !=:POST
  lambda
    /*
     * informing caller that we only support "POST"
     */
    pf.web.response.echo
      arg1:"Sorry, we only allow \"POST\" requests in here ..."

else-if:!
  :@/./-2/0/"=/multipart\\/encrypted/"?node
  lambda

    /*
     * informing caller that we only support 
     * "multipart/encrypted" content here
     */
    pf.web.response.echo
      arg1:@"Sorry, we only allow ""multipart/encrypted"" here, and 
the MIME message needs to be encrypted for ""john@somewebsite.com"""
else

  /*
   * retrieving raw bytes from HTTP request
   */
  pf.web.request.get-body

  /*
   * decrypting content
   */
  pf.crypto.pgp.decrypt:@/-/0?value
    password:foobar

  /*
   * verifying the signature of the MIME message
   */
  pf.crypto.pgp.verify-signature:@/-/1?value

  /*
   * making sure we echo our response as "multipart/mixed"
   */
  pf.web.response.headers.set:Content-Type
    source:multipart/mixed

  if:@/-2/0/0?value
    =:thomas@magixilluminate.com
    lambda

      /*
       * parsing the MIME message
       */
      pf.mime.parse-mime:@/./././*/pf.crypto.pgp.decrypt/1?value

      /*
       * verifying the caller supplied the correct parameters
       */
      if:!
        :@/./-/*/**/execute?node
        lambda
          pf.web.response.echo
            sign:john@somewebsite.com
              password:foobar
            encrypt
              :thomas@magixilluminate.com
            return-value:@"Sorry, you need to supply an ""execute"" MIME entity for me
such that I have something to do, otherwise it gets booring here ..."
              Content-Type:text/Hyperlisp
              Content-Disposition:inline; name=return-value
      else

        /*
         * appending the given Hyperlisp in our [_exe-node]
         * for the to verify that only [pf.data.insert] and/or
         * [pf.data.select] is given us.
         */
        append:@/+?value
          source:@/././-2/*/**/execute?value
        _exe-node:node:
        
        /*
         * checking that code is not malicious,
         * before we execute it!
         */
        if:@"@/-/#/*/**
  (!/pf.data.select)
  (!/pf.data.insert)
  (!/for-each)
  (!/set/""=/.*value$/m"")
  (!/set/""=/.*node$/m""(!/*/source/.))
  (!/append/""=/.*node$/m""/*/source/""=/.*node$/""(!/""=/#/m"")/.)
  (!/_data)
  (!/pf.web.widgets.create)
  (!/class)
  (!/widgets)
  (!/literal)
  (!/innerValue)
  (!/source)
  (!/element)
  (!//)
?node"
          lambda

            /*
             * code was potentially malicious
             * hence we don't allow execution of it!
             */
            pf.web.response.echo
              sign:john@somewebsite.com
                password:foobar
              encrypt
                :thomas@magixilluminate.com
              return-value:@"Sorry, we only allow [pf.data.select], 
[pf.data.insert], [for-each], [set] and [append] objects here. We found 
""{0}""
in your code, which is a potential security threat!"
                :@@/./././.?value
                Content-Type:text/plain
                Content-Disposition:inline; name=return-value
        else
          /*
           * We have now decrypted, verified the signature,
           * and made sure the given code is allowed for execution.
           * Hence, we can now safely execute the given Hyperlisp.
           */
          lambda:@/./-2?value

          /*
           * echo the results of executing the
           * given Hyperlisp back to caller.
           */
          pf.web.response.echo
            sign:john@somewebsite.com
              password:foobar
            encrypt
              :thomas@magixilluminate.com
            return-value:@/././-2/#/*?node
              Content-Type:text/Hyperlisp
              Content-Disposition:inline; name=return-value
  else

    /*
     * since this email was not signed by "thomas",
     * we return an "error" to caller
     */
    pf.web.response.echo
      return-value:"Sorry dude, I only trust \"thomas\" here ...!"
        Content-Type:text/plain

Make sure you exchange all the [sign], [password] and [encrypt] nodes with cryptography keys and/or certificates installed in your own GnuPG database. (hint; use KGpg)

Then you’ll need to create some sort of “consumer” for this Web Service. If you wish, you can create a web form, which allows you to type in any Hyperlisp, and invoke the Web Service, by using the code below;

/*
 * creating our main widget
 */
pf.web.widgets.create
  class:span-24 prepend-top info
  widgets

    /*
     * header (H1) element
     */
    literal
      class:span.24 prepend-top
      element:h1
      innerValue:"Give me some pf.lambda ..."

    /*
     * text area
     */
    literal:exp
      element:textarea
      class:span-12
      placeholder:Type in some Hyperlisp here ...

    /*
     * execute button
     */
    literal:execute
      element:button
      innerValue:Execute Hyperlisp ...
      class:clear span-5
      onclick

        /*
         * retrieves the expression given by user
         */
        pf.web.widgets.property.get:exp
          value

        /*
         * creates our web request
         * and puts the results into
         * result widget below
         */
        pf.net.create-request:"http://127.0.0.1:8080/execute-code"
          method:post
          headers
            Content-Type:multipart/mixed
          encrypt
            :john@somewebsite.com
          sign:thomas@magixilluminate.com
            password:foobar
          decryption-password:foobar
          execute::@/./-/*/exp/*/value?value
            Content-Type:text/Hyperlisp
            Content-Disposition:inline; name=execute
        pf.web.widgets.property.set:result
          innerValue:@/./-/*/result/**/return-value?value
        pf.web.widgets.property.set:result-wrapper
          visible:true

    /*
     * result of web service invocation
     */
    container:result-wrapper
      class:span-22 success prepend-top
      style:"position:relative;"
      visible:false
      widgets

        /*
         * widget to hold results from Web Service
         */
        literal:result
          element:pre

        /*
         * button to execute results
         */
        literal
          element:button
          innerValue:Execute code
          style:"position:absolute;bottom:0;right:0"

          /*
           * executes the code from our
           * result widget
           */
          onclick
            pf.web.widgets.property.get:result
              innerValue
            lambda:@/-/*/result/*/innerValue/#/*?node

When you have done, you can start playing around with it, by typing in any Hyperlisp code in the textarea of your main driver website. If you wish to execute the code where I am doing an “inner join”, reducing 6 Web Service invocations to one, you’ll first have to open up the “pf.lambda executor”, and insert your [foo-bar] and [bar-foo] items with the following script;

pf.data.insert
  foo-bar
    name:Thomas Hansen
    email:thomas@magixilluminate.com
    address
      streeet:Dunbar Rd. 12455
      state:CA
      zip:98765
  foo-bar
    name:Jane Doe
    email:jane@doe.com
  foo-bar
    name:Bob
    phone:555-415-7890
  foo-bar
    name:John Doe
    address
      street:Glen Allen Rd. 24567
      state:CA
      zip:98765
  foo-bar
    name:Frank
    email:frank@somewebsite.com
  bar-foo
    name:Thomas Hansen
    vehicle:Harley
  bar-foo
    name:Jane Doe
    vehicle:Bike
  bar-foo
    name:Bob
    vehicle:Airplane
  bar-foo
    name:John Doe
    vehicle:Car
  bar-foo
    name:Frank
    vehicle:Boat

Then you’ll have to use the following code as input in your “main driver website”;

_data
  pf.web.widgets.create
    class:span-20 success
    widgets
      literal
        element:h1
        innerValue:Names of foo-bar items ...
pf.data.select:@/*/*/foo-bar/*/name?value
for-each:@/-/*?value
  pf.data.select:@/*/*/bar-foo/*/name/"={0}"/./*/vehicle?value
    :@/././*/__dp?value
  _data
    literal
      class:span-15 prepend-2 error
      innerValue
  set:@/-/*/*/innerValue?value
    source:"Name; {0} - Vehicle; {1}"
      :@/./././*/__dp/?value
      :@/./././*/pf.data.select/1?value
  append:@/../0/*/*/widgets?node
    source:@/./-2/*?node
set:@/|/-|/-2?node

The last piece line of Hyperlisp above, just removes the [pf.data.select] and [for-each], to conserve bandwidth when returning the executed code back to caller btw.

And that is what I refer to as an “Intelligent Web Service”

Please notice though, at the time of this writing, which is 31st of March 2015, the code needed to execute the above piece of code in Phosphorus.Five is still not yet released, but can be found in the GIT repository over at GitHub.

Also notice that if you get “weird errors”, it is probably due to that you don’t have the correct private PGP key pairs installed in your GnuPG database. Use for instance KGpg on Linux to make sure you’ve got at least two private/public key pairs. One for the “consuming website”, and another for the Web Service implementation …

Ohh yeah, almost forgot 😀