Connecting Drupal Smart Content to a Marketing API
Smart Content is a module for Drupal which enables personalized content selection for anonymous and authenticated users. The module supplies the UI and logic for creating and making these selections as well as some simple browser based conditions to test, but Smart Content by itself does not provide the data needed to support them. However, there are a couple of modules in its ecosystem that support 3rd party data providers, for e.g. Demandbase and FunnelEnvy. The idea here is if your site is already using one of these data providers to record data about anonymous users, that data can be used to deliver personalized content within Smart Content. Recently, I built a connector for Marketo RTP to Smart Content; I will update this blog with a link once it is a public module. For now, however, I believe detailing how I did it can help others connect Smart Content to any 3rd party marketing API.
The entry point is first understanding what a Response from the Marketing API looks like. For example, in FunnelEnvy, there are two fundamental options: matching the ID of an Audience, or matching the ID of a Variation. In DemandBase, there are myriad dimensions in the response. In Marketo RTP, we have 6 dimensions with a number of sub-dimensions. Either way, this Response needs to be understood so we can start representing it inside Smart Content. One way to look at this response is to query the marketing API in your browser console. For example, in inspector -> console, for RTP, I would type: rtp(‘get’, ‘visitor’, function(data){console.log(data)}); and observe the results. We’ll take a step back here to discuss setting up Smart Content before continuing.
The entry point of Smart Content (SC) is the Segment Set. Administering Segment Sets is found in the Structure -> Smart Content -> Manage Segment Sets menu (once SC is installed). A Segment Set represents some generalized way you want to segment anonymous users on your site. For example, you might title a Segment Set ‘Industry’ and then within the set, create Segments that correlate to industries like ‘banking’ or ‘manufacturing’. Once you’ve created the ‘Industry’ Segment Set, press the ‘edit’ button and you should be brought to a page where you can add Segments. This brings us to the next core piece of SC: the Condition.
You’ll notice that under a Segment you have the ability to create a list of conditions. You can select “If all” (AND) are true or “If any” (OR) are true, then the segment evaluates to true, otherwise, false. SC works by iterating through these segments and checking their conditions; once a segment’s condition(s) evaluate to true, a winner has been found and SC delivers a reaction (personalized content) based on the true segment. These conditions correlate exactly to the API Response data we discussed above. So, in code, we’ll need to create a condition that matches the API we’re using.
At this point I’ll assume that you’ve created a custom module in Drupal to represent your connector; in my case I’ve named it ‘smart_content_marketo_rtp.’ Within your module, create a ‘src’ folder and a ‘js’ folder. Inside the src folder create a ‘Plugin’ folder and inside that folder a ‘Derivative’ folder and a ‘smart_content’ folder. In the ‘smart_content’ folder we’ll have a ‘Condition’ folder, and inside that a ‘Group’ folder and an option ‘Type’ folder. The end result should look like this:
The first piece of code we’ll add to this is a new PHP Class in /smart_content/ to represent our new condition. In my case I titled it ‘MarketoCondition.php’. In addition to that, we also need to add the <module>.libraries.yml file. Here we’ll configure the module’s JS files which interact with Smart Content’s backend. Since we’re creating a new condition, we’ll follow SC’s naming conventions. In the library.yml file add your version of this config:
condition.marketo_rtp:
header: true
version: 1.x
js:
js/condition.marketo_rtp.js: { }
dependencies:
- smart_content/storage
- smart_content/condition_type.standardNote the filename you created under js:, you’ll need to also add a js file with this name under your js folder (e.g. condition.marketo_rtp.js). Okay, back to the MarketoCondition.php file.
Here is where we will define our new Condition. It will look like this:
<?php
namespace Drupal\smart_content_marketo_rtp\Plugin\smart_content\Condition;
use Drupal\smart_content\Condition\ConditionTypeConfigurableBase;
/**
* Provides a Marketo condition plugin.
*
* @SmartCondition(
* id = "marketo",
* label = @Translation("Marketo"),
* group = "marketo",
* deriver = "Drupal\smart_content_marketo_rtp\Plugin\Derivative\MarketoConditionDeriver"
* )
*/
class MarketoCondition extends ConditionTypeConfigurableBase {
/**
* {@inheritdoc}
*/
public function getLibraries() {
$libraries = array_unique(array_merge(
parent::getLibraries(),
[
'smart_content_marketo_rtp/condition.marketo_rtp'
]
)
);
return $libraries;
}
}Note the definitions in the comments. The syntax here must be preserved as Smart Content reads these entries and uses them internally. Note the filepath in the ‘deriver’ assignment; go ahead and create this PHP class in your ‘Derivative’ folder as well. Make sure to change the string in the getLibraries() method so it matches your module name and your JS file config definition in libraries.yml.
What this file is doing is defining a new Condition for Smart Content; the ‘id’ key defines how it is named when it’s passed in SC, the ‘label’ key defines how it will appear to an end user and the ‘deriver’ key points to a class that will define how SC should interpret all the dimensions in the API response we discussed earlier. In essence “what conditions should be available under the Marketo id?” Finally, overriding the getLibraries() function allows us to attach our custom JS file whenever our new Condition is used. That JS file will describe how to interact with the 3rd party API that powers our new Condition.
Next, let’s move to the deriver file defined in the comment. As shown, this file will be in src/Plugin/Derivative and must match the name you put in the comment exactly. This file can be largely isomorphic to other ConditionDeriver’s from Smart Content; a good example is the Demandbase Module. The one method we will care about in creating a custom connector is the getStaticFields() method. This is where the connector will map the marketing API’s dimensions to actual Smart Content types. If you’ve checked out the Demandbase link above, you’ll see that the three basic SC types are ‘boolean’, ‘number’ and ‘textfield.’ Hopefully the marketing API’s response you’re working with fits neatly into these. When I wrote the Marketo RTP connector, the response did not, and I had to write my own custom types. This explains where the types ‘arraytext’ and ‘arraynumber’ come from in the getStaticFields() method in this connector:
protected function getStaticFields() {
return [
'abm-code' => [
'label' => 'ABM Code',
'type' => 'arraynumber',
],
'abm-name' => [
'label' => 'ABM Name',
'type' => 'arraytext',
],
'category' => [
'label' => 'Category',
'type' => 'textfield',
],
'group' => [
'label' => 'Group',
'type' => 'textfield',
],
'industries' => [
'label' => 'Industry',
'type' => 'arraytext',
],
'isp' => [
'label' => 'Internet Service Provider',
'type' => 'boolean',
],
'location-country' => [
'label' => 'Country',
'type' => 'arraytext',
],
'location-city' => [
'label' => 'City',
'type' => 'arraytext',
],
'location-state' => [
'label' => 'State',
'type' => 'arraytext',
],
'matchedSegments-name' => [
'label' => 'Segment Name',
'type' => 'arraytext',
],
'matchedSegments-code' => [
'label' => 'Segment ID',
'type' => 'arraynumber',
],
'org' => [
'label' => 'Organization',
'type' => 'textfield',
]
];
}This will look confusing at first, but if you followed the link above about RTP’s 6 dimensions you’ll see that the getStaticFields array members match exactly to these 6 dimensions. I’ve introduced a convention here for any dimension that contains a keyed array: I’ve used a ‘-’ to separate the dimension itself from the nested key. For example ‘abm-code’ and ‘abm-name.’ This dash will be necessary later when we parse out a nested key from a condition. Note that the ‘label’ key will designate the string that users of your connector see when they create a new Condition.
At this point, if you’ve created your new Condition and your Deriver files, we have one more file to add before seeing our new Condition inside Smart Content. Under the ‘Group’ folder create a new PHP class titled after your marketing API, for e.g. mine is simply named ‘Marketo.php.’ This file groups all of the new conditions you defined in your ConditionDeriver under one name. It is a very simple file:
<?php
namespace Drupal\smart_content_marketo_rtp\Plugin\smart_content\Condition\Group;
use Drupal\smart_content\Condition\Group\ConditionGroupBase;
/**
* Provides a condition group for Marketo conditions.
*
* @SmartConditionGroup(
* id = "marketo",
* label = @Translation("Marketo")
* )
*/
class Marketo extends ConditionGroupBase {
}The ‘id’ field in the comment links it to MarketoCondition.php and the ‘label’ field defines what users will see as a grouping when they create a new Condition.
At this point, if you did not create any new types in the getStaticFields() method, we can edit our Industry Segment Set, flush caches, and then press the ‘Select a condition’ drop down. You should now be able to scroll through this list and see the new grouping and the dimensions you defined in your Deriver file. If they do not appear, then one of the steps above was done incorrectly.
If you did not create any new Types, you can skip this next section. In the getStaticFields method I pasted above, you can see two new Types, ‘arraynumber’ and ‘arraytext.’ To define these new types, we’ll create two new PHP classes in the src/smart_content/Condition/Type folder: ‘ArrayNumber.php’ and ‘ArrayText.php.’ Since these two new types are depending on a more primitive type (textfield or number), I can simply extend these more primitive types. As such, my ArrayNumber.php file will look like:
<?php
namespace Drupal\smart_content_marketo_rtp\Plugin\smart_content\Condition\Type;
use Drupal\smart_content\Plugin\smart_content\Condition\Type\Number;
/**
* Provides a 'arraynumber' ConditionType.
*
* @SmartConditionType(
* id = "arraynumber",
* label = @Translation("ArrayNumber"),
* )
*/
class ArrayNumber extends Number {
/**
* {@inheritdoc}
*/
public function getLibraries() {
return ['smart_content_marketo_rtp/condition_type.array'];
}
}And
<?php
namespace Drupal\smart_content_marketo_rtp\Plugin\smart_content\Condition\Type;
use Drupal\smart_content\Plugin\smart_content\Condition\Type\Textfield;
/**
* Provides a 'arraytext' ConditionType.
*
* @SmartConditionType(
* id = "arraytext",
* label = @Translation("ArrayText"),
* )
*/
class ArrayText extends Textfield {
/**
* {@inheritdoc}
*/
public function getLibraries() {
return ['smart_content_marketo_rtp/condition_type.array'];
}
}As you can see, since these new types will use all the same operators as their primitive types, the only method we need to override is the getLibraries() method which will pass a custom JS file for evaluating the truth values of our new Types. Note that the ‘id’ field MUST match the type name you gave in your Deriver file. Make sure to add that JS file in libraries.yml and your /js/ folder. The libraries.yml definition will look like this:
condition_type.array:
header: true
version: 1.x
js:
js/condition_type.array.js: { }
dependencies:
- core/drupalI will not go into too much detail on condition_type.array.js as its unlikely most readers are defining a new Type. The key for this file is to define new functions that Smart Content will call when it encounters an ‘arraytext’ or ‘arraynumber’. These functions follow specific naming conventions, for e.g.:
Drupal.smartContent.plugin.ConditionType['type:arraytext'] = function (condition, value) {...}Drupal.smartContent.plugin.ConditionType['type:arraynumber'] = function (condition, value) {...}
Where condition is the Smart Content Condition represented in JSON and the value is the value discovered on the page for a given visitor. These functions must return boolean values. You can check out /modules/contrib/smart_content/js/condition_type.standard.js to get a better sense of these functions. Also you can message ‘plamb’ on the Drupal Slack.
If you’ve gotten this far, then we have one more file to populate. Way back at the beginning we created the file js/condition.marketo_rtp, but left it empty. In this file we will tell Smart Content what to do when it comes across a condition of type ‘Marketo.’ We’ll open this file with the following:
(function (Drupal) {
Drupal.smartContent = Drupal.smartContent || {};
Drupal.smartContent.plugin = Drupal.smartContent.plugin || {};
Drupal.smartContent.plugin.Field = Drupal.smartContent.plugin.Field || {};
...
}(Drupal));The primary function in this file will follow Smart Content’s naming conventions:
Drupal.smartContent.plugin.Field['marketo'] = function (condition) {...}Note that the text ‘marketo’ matches the id we’ve been passing around in many other files. When Smart Content evaluates a field of grouping ‘marketo’ it will execute this function. The first job this function must perform is making sure we can access the Marketing API and get a Response. It does that by constructing a Promise so it can be done asynchronously and then it returns a resolution of that Promise to the Smart Content backend that contains the relevant value. As such, our functions skeleton will look like this:
Drupal.smartContent.plugin.Field['marketo'] = function (condition) {
Drupal.smartContent.marketo = new Promise((resolve, reject) =>
{...}
});
return Promise.resolve(Drupal.smartContent.marketo).then( (value) => {
...
});
}Let’s first take a look at what’s happening inside the Promise. Here we will be checking that we can call the API and resolve the Promise with its Response:
let attempts = 0;
const interval = setInterval(() => {
if (attempts < 200) {
if (typeof rtp === "function") {
clearInterval(interval);
rtp('get', 'visitor', function(data){
if(data.results) {
Drupal.smartContent.storage.setValue('marketo', data.results);
resolve(data.results);
} else {
resolve({})
}
});
}
}
else {
clearInterval(interval);
resolve({});
}
attempts++;
}, 10);All this code is doing is running an interval function through 200 attempts of trying to resolve a given variable as a function. This structure closely models the other Smart Content modules, the main difference is that in others the function is waiting for a JS library to be available on the page vs a function resolution; this is merely an artefact of how Marketo RTP works. Once the rtp variable is recognized as a function, the code can successfully call it the way Marketo intends. The Response data can then be passed into the resolve statement to then be dealt with when the function returns.
You might wonder how the Drupal.smartContent.storage.setValue(‘marketo’, data.results); line made it into this code block. I omitted some earlier code for clarity that deals with this. Most Smart Content connectors use a browser’s Local Storage to store the results of the Marketing APIs response. This is because the metadata associated with a user in the API rarely changes. When a user returns to the site, instead of executing the setTimeout function and waiting for the rtp function to return, we can simply grab the stored values out of their browser which will always be faster.. Putting the Local Storage code back in, the code block will look like this:
Drupal.smartContent.plugin.Field['marketo'] = function (condition) {
let key = condition.field.pluginId.split(':')[1];
if(!Drupal.smartContent.hasOwnProperty('marketo')) {
if(!Drupal.smartContent.storage.isExpired('marketo')) {
let values = Drupal.smartContent.storage.getValue('marketo');
Drupal.smartContent.marketo = values;
}
else {
Drupal.smartContent.marketo = new Promise((resolve, reject) => {
//run setTimeout code
});
}
}
}So only if the ‘marketo’ key isn’t found in Local Storage is the setTimeout code run. You can get a better sense of what these methods are doing here. But wait, what is that let key = condition.field.pluginId.split(‘:’)[1]; line about? Key is used in the return statement which we will discuss next.
When Smart Content passes around condition information, its key is always the group name, ‘marketo’ appended with the key from the Deriver class that was selected in the condition. Common keys for the RTP connector would be ‘marketo:matchedSegments-code’ or ‘marketo:industries.’ The line we mentioned above, let key = condition.field.pluginId.split(‘:’)[1];, is thereby grabbing the string that occurs after the ‘:’ and storing it for use in the return statement. This key will be used to parse the Response structure that comes back from the marketing API. Now we can look at the full return statement:
return Promise.resolve(Drupal.smartContent.marketo).then( (value) => {
//All single value members and arrays containing values
if(value.hasOwnProperty(key)){
return value[key];
}else {
//All arrays of arrays
var is_array_type = marketo_testForArray(condition.field.type);
if(is_array_type) {
var refined_key = marketo_getCorrectKey(key);
if(value.hasOwnProperty(refined_key)) {
return value[refined_key];
}
}
}
return null;
});Promise.resolve().then() guarantees the code in the then block will run when the promise resolves. Since we earlier stored the key the Smart Content condition cares about, we first check if the Marketo API’s returned value simply contains that key. If so we return the value at that key back to Smart Content; simple enough. However, if the key in question is one of our custom array types, we need to detect that, split the main key from the nested key and return the correct value. The two functions called in the else block provide that functionality.
With these pieces in place, we can now test our new connector.
At this point, we would login to whichever marketing platform we’re connecting to and create a new test segment to test against our connector. In Marketo, this means going to Web Personalization -> Segments -> Create New Segment. Every Marketing platform is a bit different, but many of them have the option of creating a segment based on a url parameter which is easy for local testing. In Marketo we would add a ‘Behavioral -> Include Pages’ segment and define a URL for testing, e.g. <test-domain>/?test=yes. Under ‘domains’ we’d make sure our <test-domain> is selected and hit save. With Marketo, we can hover the segment we just created and get the ID. This is the value Smart Content will be matching against when our connector runs.
If we then open a fresh incognito window (remember the Local Storage discussion earlier?) and load <test-domain>/?test=yes we should be matched into the Marketo Segment. We can verify this by opening the console and running rtp(‘get’, ‘visitor’, function(data){console.log(data)}); again. Expanding the return structure should show our ID in ‘matchedSegments.’
To see this matching in a decision block, we would go back to Drupal and load Structure -> Smart Content -> Manage Segment Sets. If you created a Segment Set earlier, use that one or create a new one. For Marketo we will either create or re-use a condition and select Segment ID(Marketo) and paste the segment ID we just copied. Once saved and with caches flushed, we can add a decision block in Structure -> Block Layout -> Place Block, select our new segment set, choose blocks and save. In a new incognito window, we would load <test-domain>/?test=yes and see whatever block we chose.
If you have any questions you can find me @plamb on the Drupal slack chat.
