Automating Subscriptions with Paypal

We'll be discussing the following Paypal APIs here (these are the official Paypal names for those APIs):

1. HTML - the API for generating Paypal buttons (HTML forms with hidden inputs)
2. PDT  - "Payment Data Transfer" - the API for requesting transaction details after the customer has finished checkout using a Paypal button.  Your access to this API is controlled by your PDT token, which you get from your Paypal profile page.
3. API - this is the one they actually call "API" - it can be used to request payments from customers and perform various administrative actions (like issue refunds).  One frustrating limitation is that you can't directly use this API to check the status of subscriptions created using the HTML API (e.g. whether a subscription has been cancelled by the customer), but you can hack around this limitation due to the fact that you can use this API to obtain info for all transactions. You can, however use this API to obtain the status of "recurring payments" which are a newer feature, more powerful than "subscriptions" (two concepts are not compatible, it seems).  Your access to this API is controlled by your choice of (a) an RSA certificate or (b) username/password/token (they call this token "signature" for some reason).  Again, you obtain these credentials from your Paypal profile page.
4. IPN - "Instant Payment Notification" - an API which your server can use to receive callbacks from Paypal notifying you of events related to your account (e.g. customer makes a purchase, a subscription expires, etc.)  This seems to be only way to programmatically obtain information about subscriptions from Paypal, and unfortunately, if you missed the notification (e.g. because you hadn't implemented IPN on your server yet), you'll have to update your database manually.

You can set up subscriptions by programmatically generating HTML for Paypal subscribe buttons.  Here's what happens:

1. User navigates to your product page, so you render the Paypal subscribe button (it's an HTML form which you can customize; we'll refer to this as the "button form" from now on).
2. User clicks on the button, completes the order on paypal.com and is redirected back to the return URL you specified in your button form.  Paypal will attach a "tx" parameter to the query string of this URL: this is the transaction ID.
3. Your server can make a request to the PDT API and include this TX ID to obtain more info about the transaction (e.g. which item was purchased, for how much, the fees charged, etc.)

You should also set up an IPN listener on your server if you want to be notified about changes in the status of the subscription (e.g. cancellations), otherwise you'll have to go fishing for this info manually in your Paypal account's transaction history, unless you use the following hack I came up with:

Hack - Using the "API" API  to check the status of subscriptions:

1. Get the "subscr_id" field from PDT - that's the ID of the subscription (a.k.a. "recurring payment profile")
2. Invoke the TransactionSearch method of the API to get a list of all transactions by that customer's email address ("payer_email" field from PDT), for example: 
   curl https://api-3t.paypal.com/nvp -d "METHOD=TransactionSearch&EMAIL==<replace with your payer_email value>&STARTDATE=2005-12-22T08:51:28Z&VERSION=56.0&USER=<replace with your value>&PWD=<replace with your value>&SIGNATURE=<replace with your value>
3. Look for all returned transactions whose L_TRANSACTIONID* fields are equal to your "subscr_id" from step 1. You can piece together all the events regarding this subscription this way (e.g. "Created", "Canceled", etc.)

That pretty much brings you up to date with what I know to date :)

Paypal has lots of completely different APIs and it's easy to get lost in their rather disorganized documentation.  I think I now see the big picture, so I wrote this post for my own future reference and hopefully it helped you as well.