Finally, I found a free SFTP server for my file server.

http://www.k2sxs.com/silvershield/

Although it limited only one client and max 3 session on free version. However, it run as service nature and natively support file name encoding UTF8 which is enough for me.

GoodReader syncing through 3G

SilverSHielD Management Console screen

Security Settings – anti hammer support

Expert Settings – just show off how many type of SFTP / encryption method supported

Add/Edit User screen. Support Windows/AD account, command shell login

因為ESXi 5可以裝MacOS （應該支援EFI BOOT ）所以手多多走去試。

答案原來

1. ESXi 5 所指可以安裝MacOS 是指MacOS X 10.5 Server. 無法對應10.6&10.7
2. 是可以Boot EFI , 可不用Boot loader, 但ESXi 無法處理DMG。
3. 新版Mac OS 不能從IDE CD-ROM 安裝，似取消了。由於ESXi 的Virtual CD-ROM 是IDE Mode的，所以無法使用。

要解決10.7支援，要找一套叫UNLOCKER

• How to Install Retail OS X 10.6 "Snow" and OS X 10.7 "Lion" under VMware ESXi 5 – InsanelyMac Forum
• ESXi 5 Mac OS X Unlocker – InsanelyMac Forum

要安裝MacOS，最基本的方法就是用VMDirectPath 直接經Passthrough USB Flash或CD-ROM 來開機。但這個方法比較慢，大約只有4MB/S的傳輸速度。而且由Apple DMG來製作一定要用Mac機。
最快的方法是用PREBUILD USB 來安裝。

最後找到一隻已完成的VMDK，但因為來源是較舊的VMware版本，要經過vmkfsutil 修正後才能使用。

Post installation 相對實機安裝簡單，在虛擬環境只需(只能)安裝

1. VMware tools。darwin500.iso
   這個vmwaretools其實只是power state helper及修改VGA resolution。雖然不是QE／CI，但顯示反應速度在ESXi 5環境下還可接受。
2. LegacyAppleIntelPIIXATA.kext 令VM可使用IDE Virtual CD-Rom
3. Upgrade 到 Lion 10.7.2，這個Step會比較困難，因為10.7.2內的AppleLSIFusionMPT.kext 會令Mac OS 不能順利進入。解決方法有
   • a. 用Offline  install pack MacOSXUpdCombo10.7.2.dmg, 在安裝完成並提示要Reboot時，把舊有的AppleLSIFusionMPT.kext 回復來。最好可以用Kext Utiltiy 來Rebuild security及Extension cache
   • b. 用 10.7.1的安裝USB／Custom built CD開機，並在Terminal 回復AppleLSIFusionMPT.kext

   如要避免Mac OS 會使用Cache Extension, 要在Boot flag 加入 -k
   進入Mac OS後，再用Kext Utility 來Rebuild cache

最後，啟用 OS X的Sharing內的Screen Sharing ( VNC )

Links

How to Install Retail OS X 10.6 "Snow" and OS X 10.7 "Lion" under VMware ESXi 5 – InsanelyMac Forum

After few week of Mdaemon slow queue complain, 1 week of changing hardware test and 1 week of process tracing, mail queue is now stabilized.

Summarize these week study; there are few setting should help Mdaemon run faster

1. Offload Remote Queue to other server by setting delivery options [Must]
2. Use server 2008R2 with enough memory to cache more Metafile
   and calculation of enough memory = 4GB + (files count * 1KB)
   for example, total mail store with 4 million files should have 8GB or more
3. Registry tweak  (based on Performance Tuning Guidelines for Windows Server 2008 R2)
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
   NtfsDisable8dot3NameCreation=1
   NtfsDisableLastAccessUpdate=1
4. Disk defragment (no need often defragment)
5. Application disk volume alignment, Mdaemon Application ( cfilter dir), Local Queue and User folder should be on same volume
6. CPU – higher clock / better clock efficient instead of multi core
   a. we tested on 8 core 16 thread machine, it use only few thread
   b. even average load is below 50%, you will see 2.8GHz CPU perform better than 2.4GHz CPUI think Single CPU SandyBridge 2600K server will run faster than server with Dual Socket Xeon E5625

Offload Remote Queue to other server by setting delivery options

After offload Remote Queue to another Mdaemon server, you will see Incoming Queue run faster (less lockup)

Use server 2008R2 with enough memory to cache more Metafile

When huge amount of files and mail box are accessed frequently, server 2008 will cache Metafile to memory.

for example, Mdaemon folder have 4,385,755 Files, the usage of  Metafile can reach 4.7GB.

Also, more memory also can benefit windows read cache. Equallogic SAN HQ report show the read/write ratio change from 66/33 to 15/85 after system upgraded to 16GB memory. That mean 91% of read IO saved by system cache.

Application disk volume alignment

After tracing server with Sysinternals’ Process Monitor ( 3 minutes of event, data size 5.9GB!! ),

I found,

1. Mdaemon access disk in 4KB per Read/Write, it would be fine for mail sized in KB sale. However, some user send mail in Megabyte scale.
2. Mdaemon create Local Queues and Remote Queues by coping message from Inboundfor example, a email addressed to 5 local mailbox and 5 remote mail recipient with 3 different domain will do following
   5 times of ( make unique per recipient mail header + copy remaining message body in 4kB per block from Inbound to Local till EOF)
   3 times of ( copy message in 4KB per block to Remote Queue  + create *.rte to record target recipient)
3. when process Local Queue and found user have mail forwarding setting, it will create related Local Queue and Remote Queue.
   And same as Inbound Queue, working in 4KB per block pattern.
4. Local Queue will do Antivirus and SpamAssassin check, file will move to Cfilter\Temp during CFilter process
5. The final local mail will move to user mailbox

You will found that, file will  move between CFilter, Local Queue, and User Folder.
When the folder not in same disk, Mdaemon still calling filesystem MoveFileA and move between volume handled by system.
On server 2008R2, read buffer is 512KB and write block size is 64KB.

CPU – higher clock / better clock efficient instead of multi core

You will found that, upgrade from single Quad Core CPU to dual Quad Core CPU only have slightly improvement.

It is because, queue processing seems single thread (one thread scan folder, one thread move file)
However,  CFEngine, Spamassassin and  Antivirus seems running in multi-thread, so, messages waiting inside Local Queue mostly are MDAV & SpamD processed.

今日終於有時間去試Nested ESXi5。即是在ESXi5 內安裝另一隻ESXi5。

同時也試了用PXE 安裝ESXi5

預備PXE Image 我唔多講，因為已經有好幾個BLOG有完整的教學。

基本上我是參考了這個POST
http://vcp5.wordpress.com/2011/10/01/performing-scripted-installation-of-esxi-by-pxe-booting-the-installer/

只是被pfSense個DHCP設定玩了一個小時。原來pfSense的DHCP Server 已完全支援PXE Boot，返而用Additional BOOTP/DHCP Options 會出現ERROR。

設定好Network booting後，就要設定現有的ESXi5 Host。

1. 設定 allow nested hypervisors

SSH 入ESXi 修改/etc/vmware/config

在結尾加一句

vhv.allow = "TRUE"

這樣就會令nested hypervisors生效。

2. 建立GuestOS

基本設定為

• Virtual Machine Version: 8
• Guest Operating System 為Other (64-bit)
• 2GB 以上Memory
• 2 – 4 CPUs
• E1000E
• LSI Logic SAS Adapter

3. 修改 Virtual Machine Properties

建立後再修改Guest Operating System 為 VMware ESXi 5.x

CPU /MMU Visualization 要設定為 Intel EPT /AMD RVI

EPT/RVI 應該是Memory Page Remap 一種 Wiki:Extended Page Table

4. Install from PXE

Loading ESXi installer 個速度不似由TFTP下載，應該經HTTP由IIS下載。

5. 完成及測試

ESXi5 新增的e1000e Network Adapter 可以開JumboFrame

64bit Guest under Nested ESXi

如果無設定錯誤，Server 2008R2 (64bit) 的Boot DVD 應該能進入Install Windows 畫面

Others, Bad Ending

1. 如果沒有設定Guest Operating System 為 VMware ESXi 5.x 就會出現這個問題，會令到Nested ESXi 不能執行64bit Guest

2. 1GB Memory 是不能安裝ESXi 5 的

早幾日發現, 其中一部4GB RAM 的Server 2008R2 使用了95% ( Available = 200MB, Cached = 200MB )。相對其他不同服務的Server 『大食』。

由幾個方向不斷查找，找過Kernel Page、找過Application Page及Driver memory leak也找不出問題。

最後基於該Server 有2TB 好幾百萬個檔案，找了NTFS相關的資料。

答案是MFT

好似自Vista 開始就有

原來MFT會Cache 到Windows memory，而且只會有部份顯示在Cached 一項之內。

Cached = 6786MB, Used = 12,279MB -6929MB = 5350MB
如果用RamMap一看, Cached 的6786MB 大約等如 Standby 6,914,160/1024 = 6752MB
那Active 部分的Mapped File 和 Metafile應該會視為 Used, Task Manager 內 Physical Memory Usage 藍線的部份。

一個檔案在 NTFS 內大約會使用 1KB 的MFT空間，即是如果有4,000,000個檔案，全數MFT Area 應該會用4GB。

如果那些檔案及目錄被讀取過，Metafile就會load上Memory。

以下是一部讀多過寫、有4百萬個檔案的EMAIL SERVER。

Metafile Active 有3GB 會計算在Used Memory 的部份內

睇完RAMMap 才知, 自己對Task Manager 內的數字未了解得透。

Link
How NTFS Work
Sysinternals RAMMap

上前有部DELLBlade Server壞HDD，因為有pro support，有兩個(外包)DELL Engine上來幫手REBUILD  >_<"。

我問佢地如何在ESXi 下知道RAID CARD 的status。佢地就打了好幾個電話，過了半個鐘後同我講返 『現時沒有方法』…

其實在佢地上來之前，我已用MegaCLI for vmware 睇完.

在ESXi 4下裝MegaCLI好簡單，只在到LSI個WEB 下載，並且抄到ESXi 入面。再入SSH chmod 返個檔案 executable 就可以用。

但到了ESXi 5 就出事了。有個檔案不見了。

./libstorelib.so: cannot open shared object file: No such file or directory
ERROR:Could not detect controller.
Failed to get ControllerId List.
Failed to get CpController object.

找了一會也找不到libstorelib.so。最後把心一橫，由ESXi 4.1內抄過來，可以正常執行了。

LINK
MegaCLI version 8.02.16

昨日挑戰ESXi 5.0 + VT-d + PERC5i， 結果全軍覆沒。
找了很久才發現。早在ESXi 4 開始，已有人講過PERC 5i 在已開啟VT-d 功能的主板上會出現Data Corruption.

睇來張PERC 5i 應該要退休了。

Reference

http://communities.vmware.com/message/1339356#1339356

Equallogic 在星期一發放了SAN-HQ 2.2 Early Production Access，但因Support site在星期二出現問題而未能下載。
今日終於Update了新版.

這個版本加入了以下功能:

• 95th Percentile Reporting
• Support for Multiple Server Connections
• Live View (can monitor member/volume in 3 sec interval)
• Support for PS Series V5.1 Firmware features, including:
• Audit logging
• Data Center Bridging (DCB) configuration information
• Additional Data Reports
• Group Diagnostics Report
• Alerts Report
• RAID Evaluator
• Across-Group Summaries
• Event Communication Infrastructure Upgrade
• Support for the Dell EqualLogic FS7500
• Usability enhancements
• New alerts: drive firmware out-of-date notification and disk protocol mismatch.
• Improved volume type indicator.
• Temperature display settings.
• Increase syslog file size for event messages and its log directory path verification.
• SAN HeadQuarters Client startup verification dialogs, including firewall detection,
• server logs detection, and server update detection.
• Configure parameters used by the SANHQ Server

Live View

RAID Evaluator

可以用來計算在修在不同RAID TYPE 下的效能

自從有了iPad以後，試了不少收費的App。

以下是最值得買的App

Goodreader
主要功能是讀及改pdf (PDF 版iBook?)

除了讀書外，檔案管理能力也非常強
Folder, password protected folder, rename, move, search 等功能，管理幾百個檔案也不費力。

Share file方面，有http分享檔案功能，而且可以用Windows的Network Folder連接。
Get file, 可以由iTune抄入
又可以直接由網頁下載檔案
還可連上各大網上空間、Email空間、IMAP空間，而且還可以連FTP、SFTP。

Photogene
可以直接左iPad改相及上載。

可顯示EXIF

Preset effects

Crop, Rotate, Adjust

早在一個月前已開始準備在家建立一台NAS，當時還未選定用FreeNAS還是Openfiler。
剛好有個PROJECT要為ESXi準備NAS Storage，所以返復測試了兩者。

今次測試的版本
Openfiler 2.3
FreeNAS 0.7

在驅動方面, 如終是LINUX的Openfiler佔優。
因為Openfiler 的Kernel 可以在Web介面上更新，而且想自己Compile Driver 也比較易。
無論在ESX下使用VMXNET3、還是ATOM NanoITX用RTL8111C的JUMBOFRAME支援上，比起使用FreeBSD 7.2的FreeNAS好。

FreeBSD 對於Realtek 8111的jumboframe支援應該要到 FreeBSD 8.0 才能解決。

論功能，FreeNAS 佔優
兩者皆有 iSCSI Target, iSCSI Initiator, NFS, SMB, FTP

但在FTP方面, FreeNAS 佔優。
FreeNAS用proftpd 1.31, 而Openfiler只是proftpd 1.30。雖然版本號相差很少，但可用性就大有不同。
FreeNAS是可以用UTF8的. 用UTF8支援的FTP CLIENT可以見到完整的檔名。而proftpd 1.30 就會出亂碼。

還未計FreeNAS原生有Bittorrent client，基本上可以選定FreeNAS了。

RAID管理，Openfiler LVM+XFS vs FreeNAS ZFS。
在RAID管理方面，Openfiler較強

用到電腦做家用NAS，差不多一定會用RAID。

在空間擴展方面，單以WEB面介面來比較，FreeNAS 較好。因為FreeNAS能用WEB介面使ZFS不斷以Stripe方法擴展。當然Stripe不是真正的RAID。
而Openfiler的Software RAID 只有建立array及加HDD入array為HOT SPARE功能。而且新建的array也不能加到現有的share folder入面。

但Openfiler的優勢是LVM可以在console用指令擴展array.
例如把4碟RAID5加多2碟成為6碟RAID5。

返而FreeNAS的ZFS未能做到。

Finally, LINUX WIN
餘下來的問題是選擇，要功能強大的Openfiler，但沒有UTF8 FTP支援。還是要簡單好用，但RAID未能逹標的FreeNAS?

其實最簡單就要令Openfiler入面的proftpd升給就可以。

設定方法、待續..