or1ko's diary

日々を書きます

12

言語処理100本ノック 2015

12. 1列目をcol1.txtに,2列目をcol2.txtに保存
各行の1列目だけを抜き出したものをcol1.txtに,2列目だけを抜き出したものをcol2.txtとしてファイルに保存せよ.確認にはcutコマンドを用いよ.

> cut -f 1 hightemp.txt > col1.txt
> cut -f 2 hightemp.txt > col2.txt

12.hs

import System.IO as I
import System.Environment
import System.IO.UTF8 as I8

main = do
  file <- getArgs >>= return . head
  body <- I8.readFile file
  let wss = map words $ lines body
  I.writeFile "col1.txt" $ unlines $ map head wss
  I.writeFile "col2.txt" $ unlines $ map (last . take 2) wss

実行結果は省略。

11

11. タブをスペースに置換
タブ1文字につきスペース1文字に置換せよ.確認にはsedコマンド,trコマンド,もしくはexpandコマンドを用いよ.

>sed "s/\t/ /g" hightemp.txt
>tr "\t" " " < hightemp.txt

expandはなかったためやめる。

11.hs

import System.IO.UTF8 as I8
import System.Environment
import Prelude as P

main = do
  filename <- getArgs >>= return . head
  body <- I8.readFile filename
  P.putStrLn $ unlines $ map (map f) $ lines body
    where
      f '\t' = ' '
      f x = x

読み込みはutf8-stringで、出力がPreludeという組み合わせで
うまくいくのがよくわからないが、ひとまずこれで。

10

言語処理100本ノック 2015

2章はUnixのコマンドの練習のようだけれども、
Haskellを書きたいので、同様のプログラムも作成していく。と思っていたら、よく読んでみたら、プログラム書く演習だった。

10. 行数のカウント
行数をカウントせよ.確認にはwcコマンドを用いよ.

10.hs

import System.Environment
import qualified System.IO.UTF8 as I8

main = do
  file <- getArgs
  body <- I8.readFile $ head file
  print $ length $ lines body
>wc -l hightemp.txt
     24 hightemp.txt
>runghc 10.hs hightemp.txt
24

utf8-string-0.3.8使用。バージョン1では、System.IO.UTF8をGHCで対応したから削除したとのことだったが、つかっているghcHaskell Platform 2014を使っているため。

09

言語処理100本ノック 2015

09. Typoglycemia
スペースで区切られた単語列に対して,各単語の先頭と末尾の文字は残し,それ以外の文字の順序をランダムに並び替えるプログラムを作成せよ.ただし,長さが4以下の単語は並び替えないこととする.適当な英語の文(例えば"I couldn't believe that I could actually understand what I was reading : the phenomenal power of the human mind .")を与え,その実行結果を確認せよ.

09.hs

import System.Random
import Data.List

main = do
  gen <- getStdGen
  print $ unwords $ map typoglycemia $ zip (words s) (randoms gen :: [Int])
    where 
      s = "I couldn't believe that I could actually understand what I was reading : the phenomenal power of the human mind ."

typoglycemia (s, hint) 
    | length s <= 4 = s
    | otherwise     = [head s] ++ shuffle hint (init $ tail s) ++ [last s]

-- hint は文字列をシャフルする際に使う数値
shuffle hint s = tmp !! (mod hint (length tmp))
              where
                 tmp = permutations s
> runghc 09.hs
"I cun'oldt bevilee that I could autlclay udanrestnd what I was randeig : the paomehnnel peowr of the haumn mind ."
> runghc 09.hs
"I could'nt bieelve that I colud aclltuay ueandsntrd what I was reandig : the paemnhonel poewr of the hamun mind ."
> runghc 09.hs
"I c'dlunot beileve that I cluod alucltay unednrstad what I was rdianeg : the panhonemel poewr of the hmuan mind ."

shuffleが難しかった。
typoglycemia関数をString -> IO String型にするバージョンも作成してみた。

import Control.Monad
import System.Random
import Data.List

main = do
  typoglycemias <- mapM typoglycemia $ words s
  putStrLn $ unwords typoglycemias
   where 
     s = "I couldn't believe that I could actually understand what I was reading : the phenomenal power of the human mind ."

typoglycemia s 
    | length s <= 4 = return s
    | otherwise     = do
       body <- shuffle (init $ tail s) 
       return ([head s] ++ body ++ [last s])

shuffle s = 
  do 
    g <- getStdGen
    return $ tmp !! (mod (fst $ next g) (length tmp))
      where
        tmp = permutations s 

08

言語処理100本ノック 2015

08. 暗号文
与えられた文字列の各文字を,以下の仕様で変換する関数cipherを実装せよ.

英小文字ならば(219 - 文字コード)の文字に置換
その他の文字はそのまま出力
この関数を用い,英語のメッセージを暗号化・復号化せよ.

import Data.Char

main = putStrLn $ cipher $ cipher s
  where
    s = "Hi He Lied Because Boron Could Not Oxidize Fluorine. New Nations Might Also Sign Peace Security Clause. Arthur King Can."

cipher s =  map f s
  where
   f c | isLower c = chr (219 - ord c)
       | otherwise = c

逆関数が必要かと思ったけど、不要だった。
こういう関数を対合というらしい。

対合 - Wikipedia

07

言語処理100本ノック 2015

07. テンプレートによる文生成
引数x, y, zを受け取り「x時のyはz」という文字列を返す関数を実装せよ.さらに,x=12, y="気温", z=22.4として,実行結果を確認せよ.

07.hs

import Text.Printf

main = putStrLn $ f 12 "気温" 22.4

f x y z = printf "%s時の%sは%s" (show x) y (show z)
> runghc .\07.hs
12時の気温は22.4

04-06

言語処理100本ノック 2015

04. 元素記号
"Hi He Lied Because Boron Could Not Oxidize Fluorine. New Nations Might Also Sign Peace Security Clause. Arthur King Can."という文を単語に分解し,1, 5, 6, 7, 8, 9, 15, 16, 19番目の単語は先頭の1文字,それ以外の単語は先頭に2文字を取り出し,取り出した文字列から単語の位置(先頭から何番目の単語か)への連想配列(辞書型もしくはマップ型)を作成せよ.

04.hs

import qualified Data.Map as M

main = print $ M.fromList $ zip (map f $ zip (words s) [1..]) [1..]
  where 
    s = "Hi He Lied Because Boron Could Not Oxidize Fluorine. New Nations Might Also Sign Peace Security Clause. Arthur King Can."

f (s,n)  
  | elem n ns = take 1 s 
  | otherwise = take 2 s
    where
    ns =  [1, 5, 6, 7, 8, 9, 15, 16, 19]
> runghc 04.hs
fromList [("Al",13),("Ar",18),("B",5),("Be",4),("C",6),("Ca",20),("Cl",17),("F",9),("H",1),("He",2),("K",19),("Li",3),("
Mi",12),("N",7),("Na",11),("Ne",10),("O",8),("P",15),("S",16),("Si",14)]

ghciで簡単に書く方法を思いつかなかったのでファイルにした。
辞書は久しぶりに使った。

05. n-gram
与えられたシーケンス(文字列やリストなど)からn-gramを作る関数を作成せよ.この関数を用い,"I am an NLPer"という文から単語bi-gram,文字bi-gramを得よ.

Prelude> import Data.List
Prelude Data.List> let f s = filter (\x -> length x == 2) $ subsequences s
Prelude Data.List> f $ words "I am an NLPer"
[["I","am"],["I","an"],["am","an"],["I","NLPer"],["am","NLPer"],["an","NLPer"]]
Prelude Data.List> f $ "I am an NLPer"
["I ","Ia"," a","Im"," m","am","I ","  ","a ","m ","Ia"," a","aa","ma"," a","In"," n","an","mn"," n","an","I ","  ","a "
,"m ","  ","a ","n ","IN"," N","aN","mN"," N","aN","nN"," N","IL"," L","aL","mL"," L","aL","nL"," L","NL","IP"," P","aP"
,"mP"," P","aP","nP"," P","NP","LP","Ie"," e","ae","me"," e","ae","ne"," e","Ne","Le","Pe","Ir"," r","ar","mr"," r","ar"
,"nr"," r","Nr","Lr","Pr","er"]
Prelude Data.List>

06. 集合
"paraparaparadise"と"paragraph"に含まれる文字bi-gramの集合を,それぞれ, XとYとして求め,XとYの和集合,積集合,差集合を求めよ.さらに,'se'というbi-gramがXおよびYに含まれるかどうかを調べよ.

06.hs

import Data.List

main = do 
  print $ union x y
  print $ intersect x y
  print $ x \\ y
    where
      x = nub $ bigram "paraparaparadise"
      y = nub $ bigram "paragraph"

bigram = filter (\x -> length x == 2 ) . subsequences
> runghc 06.hs
["pa","pr","ar","aa","ra","pp","ap","rp","rr","pd","ad","rd","pi","ai","ri","di","ps","as","rs","ds","is","pe","ae","re"
,"de","ie","se","pg","ag","rg","gr","ga","gp","ph","ah","rh","gh"]
["pa","pr","ar","aa","ra","pp","ap","rp","rr"]
["pd","ad","rd","pi","ai","ri","di","ps","as","rs","ds","is","pe","ae","re","de","ie","se"]