Open-sourcing the first OpenRTB Scala framework
Brief introduction to the RTB world ⚡️
OpenRTB is an open source specification defining Real Time Bidding (RTB).
Real Time Bidding (RTB) is the process happening behind each online ad impression. An auction is sent in real-time to multiple buyers while the web page or the email is loading.
The whole bidding process takes only few milliseconds before displaying the right ad to the right user.
Note: If you’re impatient, you can start cloning now from our GitHub
Why Scala OpenRTB ?
Scala OpenRTB provides performant and easy-to-use JSON (de-)serialisation tools for OpenRTB entities. It also allows you to define (de-)serialisers for your custom OpenRTB extensions.
Scala OpenRTB also aims at reducing boilerplate and making your code much more elegant than using the Google Java OpenRTB API.
Long story short: if you’re using Scala and dealing with Google Java OpenRTB structures, using Scala OpenRTB may well be your best choice.
Scala-OpenRTB by example 🍿
Let’s start with a simple Bidder
During the example we will be using cats as we are really keen of it 😍
Let’s start with a simple example. We are going to build a simple Bidder capable of accepting a BidRequest and returning a number of bids aggregated in a BidResponse.
First, let’s take a look at the Bidder trait from Scala OpenRTB:
trait Bidder[F[_]] {
def bidOn(bidRequest: BidRequest): F[Option[BidResponse]]
}This trait exposes a single method bidOn, responsible for handling the bidding process, and optionally transforming any bid request to a bid response. You also have the option to not return anything, which can be useful if the bid request is not respecting your expectations. In this example we will use this trait, but you can use Scala OpenRTB without implementing it.
class Bidder[F[_] : Applicative] extends OpenrtbBidder[F] {
override def bidOn(bidRequest: BidRequest): F[Option[BidResponse]] = {
Some(BidResponse.defaultInstance)
.pure[F]
}
}That’s all we really need. Of course, this BidResponse is too simple to be valid. Let’s mock a more realistic implementation.
class Bidder[F[_] : Applicative] extends OpenrtbBidder[F] { /**
* Handle the bidding process
*/
override def bidOn(bidRequest: BidRequest): F[Option[BidResponse]] = {
val bidResponse: F[Option[BidResponse]] = bidOnImps(bidRequest.imp)
.map {
case Nil => None
case bids@_ =>
Some(BidResponse(
id = bidRequest.id,
seatbid = Seq(SeatBid(
bid = bids.toSeq,
seat = Some("seat-1")
)),
bidid = Some("1")
))
}
bidResponse
}
private def bidOnImps(imps: Traversable[Imp]): F[Traversable[Bid]] = {
???
}
}
Here we bid on a list of Imp from the bidRequest.bidOnImps is not meant to be implemented in this article, but if you want a simple example you can find the code on Github.
Now that the core of our bidder is ready, we are going to make it accessible through `http`. We choose to use http4s for this example but Scala OpenRTB is compatible with every http DSL handling JSON (you can also use Protobuf).
Let’s open our Bidder to the world 🌍
object BidderApp extends App {
stream
.compile
.drain
.runAsyncAndForget
/**
* Bind an http service
*/
def stream: fs2.Stream[Task, ExitCode] = {
import org.http4s.server.blaze._
BlazeServerBuilder[Task]
.bindHttp(port = 9000, host = "localhost")
.withHttpApp(BidderHttpAppBuilder.build())
.serve
}
}object BidderHttpAppBuilder {
private val bidder = new Bidder[Task]
private val dsl = Http4sDsl[Task]
/**
* This method provide the HttpApp we will expose with our http server
* This implementation rely on Task but you can use any effect you want
*/
def build(): HttpApp[Task] = {
import dsl._
import org.http4s.HttpRoutes
import org.http4s.circe._
import org.http4s.implicits._
/**
* bidRequest decoder required to decode a BidRequest from a json
*/
implicit val bidRequestDecoder: Decoder[BidRequest] = OpenRtbSerdeModule.bidRequestDecoder
implicit val bidRequestEntityDecoder: EntityDecoder[Task, BidRequest] = jsonOf[Task, BidRequest]
HttpRoutes.of[Task] {
case req@POST -> Root / "bid" =>
for {
bidRequest <- req.as[BidRequest]
response <- handleBid(bidRequest)
} yield response
}.orNotFound
}
/**
* Transform a BidRequest to a BidResponse
*/
private def handleBid(bidRequest: BidRequest) = {
import dsl._
import org.http4s.circe._
bidder
.bidOn(bidRequest)
.flatMap {
case Some(bidResponse) =>
// encode the bidResponse to a json object as part of the http response body
Ok(OpenRtbSerdeModule.bidResponseEncoder(bidResponse))
case None =>
// return a 200 without any entity if there is no bidReponse
Ok()
}
}
}
Here we bind an http server on localhost:9000 which replies on POST requests to /bid . We rely on the bidOn method from the bidder we implemented just before.
Notice the ease to decode and encode both the bidRequest and the bidResponse using Scala OpenRTB. All you have to do is use the relevant encoder / decoder provided by the SerdeModule. Theses coders handle the encoding and decoding of all the objects specified by the OpenRTB 2.5 specification ✨
Extending our entities 💥
The AdTech and programmatic industry is full of vendor-specific notions.
Thankfully, we can extend any part of our OpenRTB object. We start by specifying our extension using a simple .proto file.
syntax = "proto2";
import "openrtb.proto";
package com.powerspace.openrtb.example;
message BidResponseExt {
optional string protocol = 1;
}
extend com.google.openrtb.BidResponse {
optional BidResponseExt bidResponseExt = 1000;
}In this example, we extend our bidResponse with an optional extension containing an optional field, protocol .
To add this extension to our bidResponse object, we use the withExtension[T] method.
/**
* Handle the bidding process
*/
override def bidOn(bidRequest: BidRequest): F[Option[BidResponse]] = {
val bidResponse: F[Option[BidResponse]] = bidOnImps(bidRequest.imp)
.map {
case Nil => None
case bids@_ =>
Some(BidResponse(
id = bidRequest.id,
seatbid = Seq(SeatBid(
bid = bids.toSeq,
seat = Some("seat-1")
)),
bidid = Some("1")
))
}
bidResponse
.map(
_.map(withProtocol)
)
}
/**
* Add the protocol extension on the bidResponse define in example.proto
*/
private def withProtocol(bidResponse: BidResponse) =
bidResponse.withExtension(ExampleProto.bidResponseExt)(
Some(BidResponseExt(Some("define your protocol")))
)Using the same bidOn method we saw before, we simply extend our bidResponse by specifying the extension and the value we want to add to our entity.
Yet, if you launch your application and call it, you will see that the JSON you get as a response does not contain our extension. To add it, we need to register our extension in the serde module.
Defining our own SerdeModule 🛠
To register any extension for encoding or decoding, we need to define our own SerdeModule extending the base one.
object ExampleSerdeModule extends SerdeModule {
override def nativeRegistry: OpenRtbExtensions.ExtensionRegistry = ExtensionRegistry()
override def extensionRegistry: OpenRtbExtensions.ExtensionRegistry = ExtensionRegistry()
.registerExtension(ExampleProto.bidResponseExt)
}We then simply register our extension to the extensionRegistry. Scala OpenRTB will handle the rest for you. 💫
What about more complex objects ? What should I do?
Let’s take for example the following .proto file defining a “complex” extension to the Bid entity.
message Ids {
optional int32 uniqueId = 1001;
optional int32 anotherUniqueId = 1002;
optional int32 onMoreUniqueId = 1003;
repeated string multipleUniqueIds = 1004;
optional int32 lastButNotLeastId = 1005;
}message BidExt {
optional Ids ids = 1001;
}extend com.google.openrtb.BidResponse.SeatBid.Bid {
optional BidExt bid = 1000;
}
In this case, we are not able to serialize or deserialize the extension by default. Thankfully, Scala OpenRTB allows you to define your own coders and decoders 🤘
object ExampleSerdeModule extends SerdeModule {
import com.powerspace.openrtb.json.util.EncodingUtils._
implicit val idsEncoder: Encoder[Ids] = openRtbEncoder[Ids]
implicit val idsDecoder: Decoder[Ids] = openRtbDecoder[Ids]
override def extensionRegistry: ExtensionRegistry = ExtensionRegistry()
.registerExtension(ExampleProto.bid)
override def nativeRegistry: ExtensionRegistry = ExtensionRegistry()
}And that’s it for our simple introduction to Scala OpenRTB. Code from the example above and more above can be found in the Github repository.
Conclusion
Scala OpenRTB helps you build your Programmatic / RTB project in a very efficient way without the boilerplate required by the Java OpenRTB project.
Scala OpenRTB is currently deployed on production at Powerspace. Despite being a young project it has been already battle tested and it will be maintained.
Bonus: Job alert 🤓
We have an exciting job opening right now in Paris. If you love Scala, reactive programming, and playing with high traffic and latencies constraints in a modern tech environment (Scala, Kafka, ML, Kubernetes, etc.), join us now!
