Unicode is hard

7th October, 2007 . 56 comments

Full unicode support is steadily being fleshed out for rst2a's rendering API.

Since the unicode branch has been merged, unicode support in Django seems to be very solid.

However, when files are posted as a file upload, Django doesn't make their contents available as a unicode string. I'm not certain how / if to handle this at the framework level - as you don't want binary files getting encoded. There's a ticket open with regards to content transfer encoding. Perhaps something could be worked in with that.

Its easy to handle though with:

from django.utils.encoding import smart_unicode
content = smart_unicode(file['content'])

This got the unit tests passing, but Duncan could still make PDF rendering blow up by mashing alt+some keys on his macbook :(

Using rst2latex with --input-encoding=UTF-8 and --output-encoding=UTF-8 we were still getting this error:

! Package inputenc Error: Unicode char \u8:? not set up for use with LaTeX.

The fix for this was:

sudo apt-get install latex-ucs

rst2a should now handle most documents you throw at it, with regards to unicode. There are some issues with PDF when rendering odd characters. For example I'm trying to work out how to render documents with the ยข character. LaTeX gives this error:

! LaTeX Error: Command \textcent unavailable in encoding T1.

Thanks Duncan and Gabriel for your help.

56 Comments

Subscribe to comments

 #1

Forest Bond - 4th January, 2008 at 2:43 p.m.

xetex is latex that handles Unicode input (and TrueType fonts). Have you tried that?

 #3

http://www.mbttrainersshoes.com - 7th March, 2010 at 6:52 p.m.

Thanks for the great article here.
we sale mbt shoes online:
mbt trainers shoes,
mbt walking shoes,
mbt shoes sale,
mbt shoes,
mbt trainers,

discount christian louboutin
discount christian louboutin pumps
coach handbags,
coach handbag outlet

 #4

ddd - 7th March, 2010 at 7:14 p.m.

cheap ugg

nike shoes

 #5

air max 2010-3-4 - 7th March, 2010 at 8:01 p.m.

Are you prepare a pair of sport shoes this spring,the you can consider of the nike air max,such as air max 90,air max 360,air max 95 and other style shoes.The air max 2009 or air max 2010 are new brand on sale. Special nike air max 90,nike air max 360 and nike air max 95,the three styles of shoes are popular in the world. I suggest you choice the nike air 90 which shoes is hot sale in nike air max 2010,of course you can choice the cheap air max shoes or discount air max shoes.Now the air max shoes sale ,air max 90 sale,air max 360 sale,air max 95 sale are popular by younger,you can let yourself in fashion wear air max shoes .

 #6

Ccboots - 7th March, 2010 at 8:25 p.m.

Paul Smith not only in fashion, but in the wider context of popular culture. This is evident in the overall look of both Paul Smith Shoes, Paul Smith Wallet, Paul Smith Cufflinks and Paul Smith Bags.There are new brand paul smith sale online,such as cheap paul smith shoes ,cheap paul smith bags,cheap paul smith wallets,cheap pual smith cufflinks,Paul Smith is an English fashion designer, whose business and reputation is founded upon his menswear.So you should take a look of the paul smith shoes sale, paul smith bags sale,paul smith wallets sale,pual smith cufflinks sale, etc.You must be love these paul smith on sale.

 #7

wds - 7th March, 2010 at 9:06 p.m.

Can you find the replica handbags wholesale?
How to Find designer handbags on sale?
Bottega Veneta Handbags High End Designer Goods.
Why Burberry Handbags Are So Hot?
Why Cartier Handbags Are So Hot?
Why Celine Handbags Are So Hot?
Why Chanel Handbags Are So Hot?
Give My Love to Christian louboutin sale.
Why Do replica Christian Louboutin Get Termed As True Blue Blood?
Great cheap designer shoes!

 #8

coach bags - 8th March, 2010 at 12:41 a.m.

Coach are known as Elegant,coach pursesand coach handbagsare included.

 #12

wangxiaowen - 8th March, 2010 at 1:08 a.m.

=======================================================================
http://www.handbags-shop.com

handbags purses
bags cheap Handbags
designer Handbags designer purses

buy handbags buy purses
fake purses fake bag
fake handbags discount bags

=======================================================================
http://www.sellcheaphandbags.com

cheap handbag cheap purses
designer purses designer bags
designer handbags authentic handbags

replica handbags replica bags
replica purses cheap bags
================================

 #13

dfgdf - 8th March, 2010 at 1:15 a.m.

Police said Dunk SBthe septuagenarian driver, Womens Nike Dunk who was not immediately identified, lost control of his Buy Nike Dunk black 2010 Mercedes after dropping off a passenger at Cheap Nike Dunks the church.Cops said the church'sCheap Nike Dunk surveillance camera captured the tragedy Puma Shoeson tape.The video, according to cops, shows Nike Dunks the driver pulling into the parking lot and dropping off his passengerNike Dunk. The Mercedes then pulled forward about 10 http://www.alltopshoes.com/product_3.... >Nike Dunk Lowfeet before suddenly shifting into reverse, accelerating in a reverse semicircle and plowing http://www.alltopshoes.com/product_5.... >Nike Dunk Highinto the crowd standing next to the playground.The driver is related to at least one of the women injured in the accident, witnesses said."It was an accident," said aNike Air Force 1 parishioner. "He was leaving."The injured victims were being treated last night for minor injuries at Lutheran Medical Center.A member of the predominantly Korean church said his 1-year-old son seemed fine after being treated at the Air Force 1 Shoeshospital, but his wife suffered several broken fingers.The SUV is registered to a family in Roslyn Heights, L.I.Churchgoers called 911 and rushed to help the injured. The woman who took the brunt of the impact died in the parkingBape Shoes lot as tearful parishioners prayed for a miracle.The driver was not charged, police said. A law enforcement source at the scene said no drugs or alcohol was suspected.

 #14

wds - 8th March, 2010 at 2:05 a.m.

Can you find the replica handbags wholesale?
How to Find designer handbags on sale?
Bottega Veneta Handbags High End Designer Goods.
Why Burberry Handbags Are So Hot?
Why Cartier Handbags Are So Hot?
Why Celine Handbags Are So Hot?
Why Chanel Handbags Are So Hot?
Give My Love to Christian louboutin sale.
Why Do replica Christian Louboutin Get Termed As True Blue Blood?
Great cheap designer shoes!

 #23

jordan shoes - 9th March, 2010 at 1:46 a.m.

Jordan shoes have not only dominated the sports and particularly the basketball shoes, but they also have been incorporated into the music world; for instance it's not anything new to see pop music celebrities adorning the shoe. Hollywood celebrities too have not been left out.

There are a number of countless moment's celebrities, athletes, and popular music artists have been seen wearing Air Jordan shoes. For example Nike jordan shoes Brand have produced new brands of Jordan Shoes specifically designed for the hip hop music videos which have been as a form of product placement. You will be surprised by how popular Air Jordans are and how many movies and hip hop artists wear them in Hollywood.Music and famous rapper Kanye West has also been on the celebrities' lists of those seen wearing Air Jordan shoes. Kanye west has been seen wearing the jordan shoes in his most Music Videos. His favorite Jordan Shoes are the Air Jordan 8 Aqua. Kanye has been frequently spotted rocking with this pair of Jordan brand whether when performing, playing or watching his favorite sport, the NBA basketball league or walking. Carmelo Anthony a well known star player of the Denver Nuggets Has also endorsed the Nike/Jordan brand of boots.

 #24

fdgdf g - 9th March, 2010 at 4:03 a.m.

What a great post! Now let me tell you more information about http://www.famousbrandbag.com/Givench... Handbags. It is made from the best material as well as http://uggboots-euro.com/UGG-Women... Women's Handbags. The designer made a great decision to add http://www.famousbrandbag.com/Tiffany... Money Clips in its zipper chain. It will look great with all outfits.Both http://sell-ugg.com/UGG-Sundance-Limi... Sundance Limited Edition and http://uggboots-euro.com/UGG-Classic-... UGG Classic Tall Patent Paisley boots can add its chic design.

 #25

Christian Louboutin Shoes - 9th March, 2010 at 7:19 p.m.

He began to sketch it with almost any equip. Christian louboutin These are an old museum behaviors the endorse of Harper's Bazaar tacked to his childhood. Christian Louboutin Shoes These fabulous Christian Louboutin boots. "She looks so great there, doesn't she?" asks the December defend of a penknife with a big red X through it; the shoes were forbidden because the heels wrecked the parquet. Christian Louboutin Boots "After Madonna," he says incredulously, "who runs At the time, everyone was still the contend of the mules. Get prepared for American Vogue. Christian Louboutin Sandals "He brings a real, sexy Frenchness to shoe propose."

 #26

cheap louboutin - 9th March, 2010 at 7:56 p.m.

The world is cute just because of them. What will the fashion people in streets do?You See, louboutin Direct the Fashion Trend of Detroit. The present young people like to dress themselves up different, in any case just to look like different from others. In the List of Most Welcomed Embellishments, christian louboutin uk are the NO.1 Therefore, we can often see many fantastic impersonations that may be fashionable, strange, or alternative in christian shoes.Habits that Stimulate Excoriation to Christian louboutin, everyone has their own views Louboutin shoes, and then let's follow the street snaper to have a look at them in the Occident. Why Shop christian louboutin shoes for WaitressesHey Colorists! Keep Away from cheap christian louboutin!
Special skirt with big lattices is rare. Have a louboutin sale and maybe you can obtain an unexpected effect ysl shoes. The suit louboutin shoes uk upper body doesn't feel obtrusive, probably because of colour assortment is syncretic. So there can be such good effect.

 #27

christian louboutin - 9th March, 2010 at 7:56 p.m.

The world is cute just because of them. What will the fashion people in streets do?You See, jimmy choo Direct the Fashion Trend of Detroit. The present young people like to dress themselves up different, in any case just to look like different from others. In the List of Most Welcomed Embellishments, jimmy shoes are the NO.1 Therefore, we can often see many fantastic impersonations that may be fashionable, strange, or alternative in jimmy choos.Habits that Stimulate Excoriation to jimmy choo shoes, everyone has their own views jimmy choo sale, and then let's follow the street snaper to have a look at them in the Occident. Why Shop jimmy choo boots for WaitressesHey Colorists! Keep Away from jimmy choo pumps!
Special skirt with big lattices is rare. Have a jimmy choo uk and maybe you can obtain an unexpected effect buy jimmy choo. The suit discount jimmy choo upper body doesn't feel obtrusive, probably because of colour assortment is syncretic. So there can be such good effect.

 #28

louboutin sale - 9th March, 2010 at 7:57 p.m.

The world is cute just because of them. What will the fashion people in streets do?You See, louboutin Direct the Fashion Trend of Detroit. The present young people like to dress themselves up different, in any case just to look like different from others. In the List of Most Welcomed Embellishments, christian louboutin are the NO.1 Therefore, we can often see many fantastic impersonations that may be fashionable, strange, or alternative in christian shoes.Habits that Stimulate Excoriation to Christian louboutin uk, everyone has their own views Louboutin shoes uk, and then let's follow the street snaper to have a look at them in the Occident. Why Shop cheap christian louboutin for WaitressesHey Colorists! Keep Away from christian louboutin shoes!


Special skirt with big lattices is rare. Have a Christian louboutin sale and maybe you can obtain an unexpected effect louboutin sale. The suit christian louboutin shoes upper body doesn't feel obtrusive, probably because of colour assortment is syncretic. So there can be such good effect.

 #29

louboutin shoes - 9th March, 2010 at 7:58 p.m.

The world is cute just because of them. What will the fashion people in streets do?You See, louboutin Direct the Fashion Trend of Detroit. The present young people like to dress themselves up different, in any case just to look like different from others. In the List of Most Welcomed Embellishments, christian louboutin
are the NO.1 Therefore, we can often see many fantastic impersonations that may be fashionable, strange, or alternative in christian louboutin shoes.Habits that Stimulate Excoriation to Christian louboutin sale, everyone has their own views Louboutin sale, and then let's follow the street snaper to have a look at them in the Occident. Why Shop louboutin shoes uk for WaitressesHey Colorists! Keep Away from christian louboutin uk!


Special skirt with big lattices is rare. Have a Christian louboutin shoes and maybe you can obtain an unexpected effect Christian shoes. The suit louboutin shoes upper body doesn't feel obtrusive, probably because of colour assortment is syncretic. So there can be such good effect.

 #30

wds - 9th March, 2010 at 8:48 p.m.

Where Can I Find a Rating on Which replica Christian LouboutinShoes Are the Best From the Rest?
Discover Why Marc Jacobs Handbags Are Of The Highest Quality.
Where to Buy Wholesale Prada Handbags?
Roberto Cavalli Handbagsis a designer whose pieces are instantly recognizable.
Where to Buy the Best cheap led online.
Here are the world's bestwholesale led lights.
Who Wore Christian Louboutin Pumpsat the Grammy Awards This Year?
Comfortable and Fashionable Men's winter boots.
replica Tiffany Can Make One Always Feel Special.
Here are the lowestcheap abercrombie.
UGG Australia Womens Highkoo Are the Perfect Ones
UGG Australia Sundance II For Everyone.
Find Luxurious Casual Wear at theAbercrombie Men Jeans.
Find Luxurious Casual Wear at theAbercrombie Women Jeans.
Tiffany & Co. Accessories - The Emphasis is on Style.
Tiffany & Co. Pendants is Becoming a New Favorite For Women.

 #31

fae - 9th March, 2010 at 9:46 p.m.

Look forward for a proper brand!

 #32

golfequipment - 9th March, 2010 at 11:50 p.m.

There are two golf clubs store,include kinds of golf equipment,
wholesale golf

 #33

ghd - 10th March, 2010 at 1:34 a.m.

ghd is being sold at low prices. you will get Hair Straighteners, try to keep a piece of GHD Kiss .

 #34

GHD - 10th March, 2010 at 1:40 a.m.

ghd hair straighteners and chi flat iron are hot sale.ghd is a good choice.

 #35

FFFFFF - 10th March, 2010 at 5:31 a.m.

How do you like the replica watches ? A good Fake watches to a man not only the machine to recognize the time but also the symble of the Status, and the replica Longines watches to the women is also a good Accessories, just have a try to our website www.topswisswatcheszone.com and www.perfect-watches.com to choose a Replica Swiss Panerai watches, you will get a most luxury Rolex Daytona Watches at very low price. Just have a try you will be surprised.replica Omega watches
replica IWC watches
Breitling replica watches
Panerai Replica watches
replica Breguet watches
Tag Heuer Link Watches
replica cartier watches

 #37

mbt - 10th March, 2010 at 10:30 a.m.

A great kind of shoes-MBT shoes sale. The shoes are designed to simulate walking in sand. Our MBT Chapa sale is meant to make the core strengthening muscles more active, MBT shoes creating better posture.

 #38

herve leger - 10th March, 2010 at 10:30 a.m.

You will never regret to buy this Herve Leger Dress.


White Black Herve Leger Strapless on the arrangement of the clothing line is very careful.

 #39

ghd pink - 10th March, 2010 at 10:31 a.m.

Nice Hair Straighteners Ghd,you should own it.Buy a pink ghd is your best choice.

 #40

Lacoste shoes - 10th March, 2010 at 10:31 a.m.

Buy Lacoste shoes online is your right choice,lacoste men shoes sells the best around the world.

 #41

timberlandshoes - 10th March, 2010 at 10:32 a.m.

Timberland boots sale online are both good in quality and reasonable in price,don't miss this chance.The most popular one is the timberland boots


 

 #42

fav - 10th March, 2010 at 7:21 p.m.

he online auction house must pay louis vuitton 200,000 euros, or $275,200, in damages plus 30,000 euros, or $41,300, louis vuitton bags for saleIn the case of any future violations, pay lv handbag a 1,000-euro, or about $1,400, louis vuitton outlet online to an LVMH spokeswoman. In a louis vuitton handbags on sale, eBay said it was disappointed but noted louis vuitton was awarded less than the 1.2 million euros, or $1.7 million, it sought.

 #43

Jordan Shoes - 10th March, 2010 at 9:11 p.m.

Jordan Shoes, or simply Jordans are a brand of shoes produced by Nike originally designed for and endorsed by professional NBA basketball player Michael Jordan.Jordan 1 was designed by Peter Moore and released in 1985,As the first design of jordans,Air Jordan 1 was considered a breakthrough in basketball shoe design.The Jordan 3 was designed by Tinker Hatfield. Air Jordan III was the first Air Jordan to feature a visible air unit on the heel.The Air Jordan XI model was designed by Tinker Hatfield. While Jordan was still pursuing a career in baseball, Hatfield designed the shoe in hopes that Michael would eventually wear it if he returned to the NBA.The Air Jordan 11 was meant to stand out with a fusion of performance and style.

 #44

christian louboutin - 10th March, 2010 at 9:11 p.m.

Why there are so many women enjoy in christian louboutin pumps? Why obsession the world of christian louboutin?? Perhaps because christian louboutin shoes shaped women's posture; perhaps because christian shoes given women the magic of rebirth.you may choose christian louboutin.

 #45

jeans - 10th March, 2010 at 9:38 p.m.

Seven jeans are about sexy, adventurous, trendsetting apparel and accessories. Shop from our entire line of True Religion Mens.And latest designer True Religion Jeans here, welcome to buy.

 #46

Louboutin Shoes - 10th March, 2010 at 9:44 p.m.

Great article,thanks for share. Special red sole is Louboutin Shoes,which loved by women around the world. Christian louboutin pumps with free shipping are provided in Louboutin Sale online store. If disturbed, please forgive me!

 #48

lacoste shoes - 10th March, 2010 at 10:02 p.m.

Sport fans look at here,lacoste shoeson sale.Cheap Men's Lacoste Prep CB Shoes 34% off,free shipping!It's all at lacoste sale!

 #49

GHD - 10th March, 2010 at 11:22 p.m.

ghd hair straighteners and Farouk CHI are in stock, ghd is 64% off.

 #50

dfs - 11th March, 2010 at 1:16 a.m.

The national GOP is a party Buy Casual Shoes of exclusion and litmus tests, dominated casual shoes wholesale on social issues by the religious right, with zero discernible outreach by the national party to casual shoes anyone who doesn't fit neatly within its parameters. XJKEJIJFKLJ

 #51

ff - 11th March, 2010 at 1:17 a.m.

The bottom line then is this: if the fashion ghd Tea Party movement has a "narrative of impending ghd online tyranny" running through it, then there is something ghd sale objectively crazy going on within the Tea Party movement.XJKEJIJFKLJ

 #52

dsfss - 11th March, 2010 at 1:17 a.m.

Well, it turns out it was all just a fever dream, as mbt shoes Leno himself pointed out at the start of his return to "The Tonight Show," which led off with a sepia-toned "Wizard of Oz" homage. "There's discount mbt shoes no place like home," muttered Leno. Roll credits -- and poof: There he cheap mbt shoes was, striding into a crowd of fervent, bowing well-wishers, with a fresh haircut and admittedly "a little nervous." XJKEJIJFKLJ

 #53

asdf - 11th March, 2010 at 1:52 a.m.

bvlgari replica, dior jewellery,miss dior cherie, vintage jewellry,vintage cufflinks, earring jewellery,
chanel earrings, necklace jewellery,personalized necklace, bracelets jewellery,engravable bracelets,
personalised jewellery,personalised keychains, ring jewellery,
vintage rings, cartier jewellery,cartier engagement rings, gucci jewellery,gucci mane jewelry,
links jewellery, hot links, bracelet jewellery,
handmade bracelets, pendant jewellery,tiffany style pendant lights,
jewellery silver,silver cuff bracelets,
rhinestone jewellery,rhinestone clutch, wholesale jewellry,
wholesale jeweler, fashion costume jewellery,coach fashion, rings jewellery,fake plug earrings,
imitation jewellery low-priced and high quality replica jewellery.

 #54

DNF - 11th March, 2010 at 4:48 a.m.

MBT Shoes on sale, all Cheap Mbt Shoes at lowest price. Women Shoes Are Provided In Our MBT Shoes Store, more Discount Mbt shoes on sale.
Mbt2you.com is the best MBT Shoes supplier, lots Discount Mbt sale, free and fast shipping, Save 30-60% Off! High Quality, But Cheap Pice!
MBT shoes sale at Our Mbt Website, Buy Mbt shoes for you. Cheap Mbt Shoes manufacturers directory - over 2000000 registered importers and exporters. Find and Buy Mbt shoes. Mbt2you.com can help you in your search for Shoes by providing you with reviews and low prices! MBT sale online, You can find your best MBT Shoes here!
yejianping

 #55

cheap nfl jerseys - 11th March, 2010 at 5:39 a.m.

Dear friends from all over the world,We are a professional and reliable supplier of series of innovative,wholesale nfl jerseyand nfl reebok replica jerseyIve written a letter ofcheap adidas nba jersey,The Cincinnati Bengals of NFL receiver is being fined for his latest stunt donning a poncho and sombrero after scoring a touchdown this weekend a person in the NFL wholesale nba jersey,by the NFL for pretending to bribe an official a dollar.Whimlb jersey wholesale,was being reviewed during Cincinnatiwholesale reebok nhl jersey.Dear NFL League Office being great means having fun n celebrating. apology to yall. This fine comes about a month after Ochocinco was docked and on Nov. 8 Ochocinco held a dollar bill in his right hand at his official,We supply all kinds of jerseys and discount nhl hockey jerseys reprimanded le one of his catches 17-7 victory over Baltimorewith drew brees jersey andside but give it to the New Orleans Saints jersey,welcome to buy and enjoy the greatest benefits. http://www.onnhl.com/

 #56

vwgew - 11th March, 2010 at 7:41 a.m.

nfl is my favorite!

Post a comment