cl-curl

cl-curl is an interface to libcurl, a multiprotocol file transfer library. It is useful for network communication where trivial-http is too trivial. For example, it will allow you to access a web page where keeping cookies is required.

This code should be considered alpha. It works correctly using the libcurl "easy" interface and HTTP, anything else is yet to be tested. Testing, code, and documentation are gratefully accepted.

This software is distributed under the LLGPL.

As mentioned, trivial-http is recommended for simple HTTP transactions. It requires no external library.

• cl-curl-announce, announcements related to cl-curl
• cl-curl-devel; development of cl-curl. This is the list to ask questions, post suggestions, etc.

You will need

• libcurl
• UFFI
• ASDF

If you are running Debian, the following should work:

apt-get install libcurl2-dev cl-uffi cl-asdf

Once your CL implementation is running, do

(asdf:operate 'asdf:compile-op :curl)
(asdf:operate 'asdf:load-op :curl)

or, if using common-lisp-controller,

(clc:clc-require :curl)

At the present time, usage of the libcurl library is limited to the "easy" interface, as opposed to the "multi" interface. In addition, since callbacks are not defined in UFFI, no functions may be passed into the libary. For setting options, #'set-option (curl_easy_setopt) can take only strings or integers (long). General string reading and writing functions are defined in the cl-curl C library, and they can be used to send and receive strings to the URL.

Here is an example that is used successfully with a real web site (name has been changed to prevent gratuitous hammering):

(curl:with-connection-returning-string (:cookies t)
  (curl:set-option :url "http://www.example.org/perl/login.pl")
  (curl:set-option
   :postfields
   "username=myname&password=mypasswd&_submitted=1&_sessionid=&_submit=Submit")
  (curl:perform)
  (curl:set-option :url "http://www.example.org/perl/id_query.pl")
  (curl:set-option
   :postfields
   "ids=8820&timeframe=latest&sort=catnum&ascii=yes&common_name=yes&_submitted=1&_sessionid=&_submit=Submit")
  (curl:perform))

which returns the HTML from the last page as a string.

For remaining documentation, see the CL source and libcurl documentation. Some advice of mechanizing web access: Go through a manual session on a browser that will show you page source. Look for all "input" forms; these must be sent in the :postfields form. Alternatively, try a wire sniffer like Ethereal and see what you're sending out when you click things.

cl-curl has been tested with SBCL on Debian i386 and amd64 and with SBCL and OpenMCL on MacOS X.

You may browse or download the Subversion repository at svn://common-lisp.net/project/cl-curl/subversion/trunk. For example,

svn list svn://common-lisp.net/project/cl-curl/subversion/trunk
svn checkout svn://common-lisp.net/project/cl-curl/subversion/trunk

You can also get a tgz file here.