November 08, 2010

Filter Bad Words in a SQL Server 2008 Database

If you are using SQL Server 2008, you can use Full-Text Search to filter bad words in a database. The procedure to do so is by creating Stopwords and Stoplists.

Stopwords - Commonly occurring strings that do not help the search and are discarded by SQL Server.

Stoplists - A stoplist is a list of stopwords that, when associated with a full-text index, is applied to full-text queries on that index.

Note: Full-Text Search must be enabled on your SQL Server for this to work (How to Detect If Full-Text Search is Enabled on a SQL Server Database)

You can create Language specific Stopwords. For eg: you can create a Stopword in a Stoplist for Spanish.

-- Create a StopList

-- Add a Stopword to the StopList
ADD 'mui' LANGUAGE 'Spanish';

-- View the StopWord
SELECT * FROM sys.fulltext_stopwords

If you want to view all the default Stopwords present in your database in all Languages, fire this query:

SELECT * FROM sys.fulltext_system_stopwords

OUTPUT (Sample)


About The Author

Suprotim Agarwal, ASP.NET Architecture MVP works as an Architect Consultant and provides consultancy on how to design and develop Web applications.

Suprotim is also the founder and primary contributor to DevCurry, DotNetCurry and SQLServerCurry. He has also written an EBook 51 Recipes using jQuery with ASP.NET Controls.

Follow him on twitter @suprotimagarwal

No comments: