Avito.ma Hidden Ads Listing JSON API

Avito.ma is one of biggest players in classified ads field here in Morroco. As a result, I was interested in analyzing their data and possibly offer a useful service to its users.

As first try, I headed toward a classic solution web screen scraping. it worked fine. I used Python Scrapy framework to crawl their sitemap. Nevertheless, it looked like a hacky way to do it. Looking into their mobile app will strongly refer you to the idea of using some kind of an API. so, here we go:

Decompiling Avito.ma Android application

Decompiling their app APK file nowadays is an easy task. there are plenty free tools like javadecompilers.

You can download source code here

avito.ma android source code

Exploring the source code

Tracking down HTTP requests source lead us to ~\se\scmv\morocco\models\ListingQuery.java and ~\se\scmv\morocco\requests\ListingRequest.java the most important parts to look into are:
API URL endpoint

private static final String PATH = "/lij";  

Query string keys and valid values

public ListingRequest buildRequest() {  
        String caller;
        ListingRequest listingRequest = new ListingRequest();
        listingRequest.setParam("fullad", Integer.valueOf(1));
        if (!TextUtils.isEmpty(this.query)) {
            listingRequest.setParam("q", this.query);
        }
        int region_id = this.locationInfo.region.getId() > 0 ? this.locationInfo.region.getId() : 11;
        if (BUY_ADTYPES.contains(getCurrentAdType())) {
            caller = new StringBuilder(String.valueOf(region_id)).append("_").append(AdParamFieldList.AD_TYPE_K).toString();
        } else {
            caller = new StringBuilder(String.valueOf(region_id)).append("_").append(Category.DEFAULT_ST).toString();
        }
        switch ($SWITCH_TABLE$se$scmv$morocco$models$City$SearchWidth()[this.locationInfo.city.getWide().ordinal()]) {
            case Value.TYPE_FIELD_NUMBER /*1*/:
                listingRequest.setParam("m", Integer.valueOf(this.locationInfo.city.getId()));
                break;
            case Value.LIST_ITEM_FIELD_NUMBER /*3*/:
                listingRequest.setParam("w", XitiConfig.XITI_SITE_PAGE_TYPE_LIST);
                listingRequest.setParam("ca", caller);
                break;
            case Value.MAP_KEY_FIELD_NUMBER /*4*/:
                break;
            default:
                listingRequest.setParam("w", XitiConfig.XITI_SITE_PAGE_TYPE_ADVIEW);
                break;
        }
        listingRequest.setParam("w", Integer.valueOf(region_id + 100));
        listingRequest.setParam("ca", caller);
        if (this.page > 1) {
            listingRequest.setParam("o", Integer.valueOf(this.page));
        } else {
            listingRequest.delParam("o");
        }
        listingRequest.setParam("cg", Integer.valueOf(this.categoryId));
        listingRequest.setParam("st", getCurrentAdType());
        if (this.orderByPrice) {
            listingRequest.setParam("sp", Integer.valueOf(1));
        } else {
            listingRequest.delParam("sp");
        }
        for (Entry<String, Set<AdParam>> entry : this.selectedAdParamValues.entrySet()) {
            String qs = (String) entry.getKey();
            if (!qs.equals(AdParamFieldList.AD_TYPE_SWITCH_QS)) {
                for (AdParam paramValue : (Set) entry.getValue()) {
                    if (qs.equals(AdParam.PRICE_MIN) || qs.equals(AdParam.PRICE_MAX)) {
                        String paramName;
                        if (qs.equals(AdParam.PRICE_MIN)) {
                            paramName = "spr";
                        } else {
                            paramName = "mpr";
                        }
                        listingRequest.addParam(paramName, paramValue.getValue());
                    } else {
                        listingRequest.addParam(qs, paramValue.getKey());
                    }
                }
            }
        }
        return listingRequest;
    }

As we can see this means that the API endpoint for listing requests is http://www.avito.ma/lij in addition to optional query string parameters mentioned in the second snippet

Query String Param. Description
fullad fetch all ad item fields like phone number and images
q search keyword
o page number
sp order by price
spr set min price filter
mpr set max price filter
cg category ID (please refer to table below)
w region ID (please refer to table below)
ca caller = new StringBuilder(String.valueOf(region_id)).append("_").append(AdParamFieldList.AD_TYPE_K).toString();
st getCurrentAdType

example: GET https://www.avito.ma/lij?o=2&q=iphone&mpr=2500

{
  "ads_per_page":20,
  "extracted_ads":20,
  "total_ads":15370,
  "list_ads":[
    {
      "id":19825774,
      "ad_date":"Aujourd'hui, 11:24",
      "subject":"Kit iPhone 6s Neuf",
      "thumb":"http://www.avito.ma/thumbs/87/8718499725.jpg",
      "price":"200",
      "category":"5010",
      "city":"",
      "type":"Offre",
      "url":"http://www.avito.ma/vij/19825774.htm"
    },
    {
      "id":19825760,
      "ad_date":"Aujourd'hui, 11:24",
      "subject":"iPhone 5C 16GB",
      "thumb":"http://www.avito.ma/thumbs/87/8758315772.jpg",
      "price":"1.250",
      "category":"5010",
      "city":"",
      "type":"Offre",
      "url":"http://www.avito.ma/vij/19825760.htm"
    }
    ...
  ]
}