Platform API
Filter And Moderation

Filter and Moderation

To protect your users from harmful language and unsafe content, Sendbird provides a set of filters and a variety of moderation methods. The filters are designed to detect with precision anything that is seemingly inappropriate in user-submitted messages, images, and so on. The moderation methods help you reduce the need for human reviews and submission approvals. With the supported filters and moderation methods, you can determine the level of suitability of language and content for your Sendbird application.


Domain filter

The domain filter allows you to set the domains to be detected in URL-containing text and file messages as well as users’ profile images. It will filter the detected domains according to your policies and criteria.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

domain_filter

nested object

A domain filter configuration to filter out text and file messages with URLs that contain the domain set. This also filters out a user's profile image URL if it matches the domain set.

domain_filter.domains[]

array

Specifies an array of domains to detect. Each item of the array is specified at least with a combination of domain name and TLD (top level domain) like 'amazon.com'.

domain_filter.type

int

Determines which filtering mode to apply to messages with URLs that contain any of the domain set. Acceptable values are limited to the following:
- 0 (none): takes no action on matching messages. This is the default setting.
- 1 (allow): only allows messages containing URLs that match the domains property.
- 2 (block): blocks messages containing URLs that match the domains property.
- 3 (replace): detects and replaces URLs that match the domains property with asterisks (*).

domain_filter.should_check_global

boolean

Determines whether to apply the domain filter of the global application settings in addition to the settings above for the custom channel type. For the application, the property value is always set to false. (Default: false)

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning on the filter at an application-wide level 
{
    "domain_filter": {
        "domains": ["casinobetting.com", "betonhorserace.com"],
        "type": 2 
    }
}
Light Color Skin
Copy
# Request body example for turning on the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "domain_filter": {
        "domains": ["esqn.com", "sports.yaho.com"],
        "type": 2,
        "should_check_global": true
    }
}

If you want to turn off the domain filter, send a PUT request with the blank domains property like below:

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning on the filter at an application-wide level 
{
    "domain_filter": {
        "domains": []
    }
}
Light Color Skin
Copy
# Request body example for turning on the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "domain_filter": {
        "domains": []
    }
}

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.


Profanity filter

The profanity filter allows you to configure which profanity words or patterns to be detected in text and file messages, and determine how to process them according to your policies and criteria.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

profanity_filter

nested object

A filter configuration on certain words and patterns for matching character combinations in strings, which are not allowed to be used within the application.

profanity_filter.keywords[]

array

Specifies an array of words to detect. *word filters all words that end with "word" including "word" itself while word* filters all words that start with "word" including "word" itself.

profanity_filter.regex_filters[]

list

Specifies a list of regular expressions used for detecting. Each item of the list is specified in {"regex": "a pattern in regular expression"} format.

profanity_filter.type

int

Determines which filtering method to apply to messages that contain the specified keywords or regular expressions. Acceptable values are limited to the following:
- 0 (none): takes no action on matched messages. This is the default setting.
- 1 (replace): detects and replaces words that match the keywords property with asterisks (*).
- 2 (block): prevents users from sending messages that contain the keywords property or match the regex_filters property.

profanity_filter.should_check_global

bolean

Determines whether to apply the profanity filter of the global application settings in addition to the settings above for the custom channel type. For the application, the property value is always set to false. (Default: false)

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning on the filter at an application-wide level 
{
    "profanity_filter": {
        "keywords": "ass,snatch,sperm",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(fuck|prick)[^!@#$%^&*]*"
            }, 
            {
                "regex": "(http://|https://)?(casino|sex)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2   
    }
}
Light Color Skin
Copy
# Request body example for turning on the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_filter": {
        "keywords": "idiot,dummy,damn",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(bitch|assole)[^!@#$%^&*]*"
            }, 
            {
                "regex": "(http://|https://)?(casino|sex|betting)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2,
        "should_check_global": true
    }
}

If you want to turn off the profanity filter, send a PUT request with the blank keywords and regex_filters properties like below:

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning off the filter at an application-wide level 
{
    "profanity_filter": {
        "keywords": "",
        "regex_filters": []
    }
}
Light Color Skin
Copy
# Request body example for turning off the filter only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_filter": {
        "keywords": "",
        "regex_filters": []
    }
}

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.


Profanity-triggered moderation

Based on the profanity filter, this feature moderates the users who are sending profanity words to a channel. You can apply different profanity-triggered moderation to appllication-wide and channels with a custom type separately. You can also adjust the level of the moderation by configuring the number of violation limit, the time window for counting violations, and the type of moderation penalty.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

profanity_triggered_moderation

nested object

A moderation configuration on which penalty is automatically imposed on users who reach the profanity violation limit within a channel.

profanity_triggered_moderation.count

int

Specifies the number of profanity violation limit which then imposes a penalty on a user if reached. A value of 0 indicates that automatically-triggered moderation is turned off. A value of equal to or larger than 1 indicates that the moderation is turned on and imposes a penalty on a user who commits a number of violations equal to or beyond the set value of the count property within the set time of the duration property.

profanity_triggered_moderation.duration

int

Specifies the duration of the time window in seconds which counts the number of a user’s violations within a channel. For example, if the count property is 2 and the duration property is 5, the number of violations equal to or beyond 2 will be moderated for every 5 seconds. The maximum value is 86400 which indicates 60 days. (Default: 1 second)

profanity_triggered_moderation.action

int

Determines the type of moderation penalty within a channel which is permanently imposed on users until canceled. Valid values are 0 (no action), 1 (mute), 2 (kick), and 3 (ban). (Default: 0)

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning on the moderation at an application-wide level 
{
    "profanity_filter": {
        "keywords": "ass,snatch,sperm",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(fuck|prick)[^!@#$%^&*]*"
            },
            {
                "regex": "(http://|https://)?(casino|sex)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2
    },
    "profanity_triggered_moderation": {
        "count": 5,
        "duration": 5,
        "action": 1
    }
}
Light Color Skin
Copy
# Request body example for turning on the moderation effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_filter": {
        "keywords": "idiot,dummy,damn",
        "regex_filters": [
            {
                "regex": "[^!@#$%^&*]*(bitch|assole)[^!@#$%^&*]*"
            },
            {
                "regex": "(http://|https://)?(casino|sex|betting)+([-.]{1}[a-z0-9]+)*.[a-z]{2,5}(:[0-9]{1,5})?(/.*)?"
            }
        ],
        "type": 2
    }
    "profanity_triggered_moderation": {
        "count": 2,
        "duration": 10,
        "action": 2 
    }
}

If you want to turn off the profanity-triggered moderation, send a PUT request with the count property of 0 like below:

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning off the moderation at an application-wide level
{
    "profanity_triggered_moderation": {
        "count": 0  
    }
}
Light Color Skin
Copy
# Request body example for turning off the moderation only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "profanity_triggered_moderation": {
        "count": 0  
    }
}

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.


Image moderation

Sendbird’s image moderation feature is powered by Google Cloud Vision API. This feature is for moderating the text and file messages with explicit images or inappropriate image URLs, and uses five categories such as adult, spoof, medical, violence, and racy.

Image moderation is one of Sendbird's premium features, contact our sales team for further assistance.

HTTP request

Light Color Skin
Copy
// The application-wide settings.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_global

// The settings only applied to channels with a custom channel type.
PUT https://api-{application_id}.sendbird.com/v3/applications/settings_by_channel_custom_type/{custom_type}

Parameters

The following table lists the parameters that this action supports.

Parameters
RequiredTypeDescription

custom_type

string

Specifies the custom channel type to apply a set of settings to channels with the corresponding type.

Request body

The following table lists the properties of an HTTP request that this action supports.

Properties
OptionalTypeDescription

image_moderation

nested object

A moderation configuration on inappropriate images within the application. Google Cloud Vision API is used for image moderation and supports many types of images.

image_moderation.type

int

Determines the moderation method which is applied to the images and image URLs in the text and file messages. Acceptable values are limited to the following:
- 0 (none): no moderation is imposed. This is the default setting.
- 1 (normal): the messages with images or image URLs are blocked if the images or image URLs violate the content policies.
- 2 (strict): the file messages with no images are also blocked in addition to the messages with explicit images or inappropriate image URLs.

image_moderation.soft_block

boolean

If true, the moderation method set by the type property above is ignored and no moderation is imposed on the text and file messages in regard to explicit images or inappropriate image URLs. It will only give the image analysis results in the response. If false, the image moderation works according to the moderation method already set.

image_moderation.limits

nested object

A set of features pertaining to the images for moderation. Each feature specifies the content likelihood for the image which is used as a moderation standard. Acceptable likelihood values are 0 (unknown), 1 (very unlikely), 2 (unlikely), 3 (possible), 4 (likely), and 5 (very likely).

image_moderation.limits.adult

int

Specifies the adult content likelihood for the image.

image_moderation.limits.spoof

int

Specifies spoof likelihood.

image_moderation.limits.medical

int

Specifies likelihood that the image is a medical image.

image_moderation.limits.violence

int

Specifies likelihood that the image contains violent content.

image_moderation.limits.racy

int

Specifies likelihood that the image contains racy content.

image_moderation.check_urls

boolean

Determines whether to check if the image URLs in the text and file messages are appropriate.

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning on the moderation at an application-wide level 
{
    "image_moderation": {   // For more information, see the 'Properties' table above.
        "type": 1,  // The 'normal' moderation method.
        "soft_block": false,
        "limits": {
            "adult": 3,
            "spoof": 5,
            "medical": 5,
            "violence": 3,
            "racy": 4
        },
        "check_urls": true  // Check if the image URLs in messages are appropriate.
    }
}
Light Color Skin
Copy
# Request body example for turn off the moderation only effective to channels with a custom channel type
{
    "custom_type": "Sports",    
    "image_moderation": {   // For more information, see the 'Properties' table above.
        "type": 2,  // The 'strict' moderation method.
        "soft_block": false,
        "limits": {
            "adult": 4,
            "spoof": 5,
            "medical": 5,
            "violence": 5,
            "racy": 5 
        },
        "check_urls": true  // Check if the image URLs in messages are appropriate.
    }
}

If you want to turn off the image moderation, send a PUT request with the type property of 0 like below:

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Request body example for turning off the moderation at an application-wide level
{
    "image_moderation": {
        "type": 0
    }
}
Light Color Skin
Copy
# Request body example for turning off the moderation only effective to channels with a custom channel type
{
    "custom_type": "Sports",
    "image_moderation": {
        "type": 0
    }
}

When a message has passed through image moderation, Sendbird server sends back a success response body like the following:

application-wide
channels-with-custom-type
Light Color Skin
Copy
# Response body example when a message has passed through image moderation
{
    ...
    "moderation_action": 1,
    "moderation_info": {
        "action": 1,
        "message": "Success",
        "scores": { // Image analysis results.
            "adult": false,
            "spoof": false,
            "medical": false,
            "violence": false,
            "racy": false
        },
        "cache_hit": false
    },
    ...
}
Light Color Skin
Copy
# Response body example when a message has passed through image moderation
{
    "custom_type": "Sports",
    ...
    "moderation_action": 2,
    "moderation_info": {
        "action": 2,
        "message": "Success",
        "scores": { // Image analysis results.
            "adult": false,
            "spoof": false,
            "medical": false,
            "violence": false,
            "racy": false
        },
        "cache_hit": false
    },
    ...
}

When a message has been blocked by image moderation, Sendbird server sends back an error response containing the 900066 (ERROR_FILE_MOD_BLOCK) or 900065 (ERROR_FILE_URL_BLOCK) code.

Response

If successful, this action returns the updated settings of an application or a custom channel type in the response body.