2017. január 3., kedd

Agressively reducing WinSXS with DISM

In my recent quest to reduce virtual server images I noticed that a huge part of the total image is taken up by the WinSXS folder. Here is a comprehensive list of DISM commands you can run to reduce that beyond what is possible by running "Disk Cleanup" from the properties of the C: drive.

Always use a command prompt that is run as Administrator (start menu, type CMD, right click on cmd.exe, Run as Administrator)!

for Windows 8.x, 10 and Windows Server 2012 (R2)

It's always a good idea and can avoid a lot of strange error messages to make sure the Deployment Image is in a healthy condition:
Dism /Online /Cleanup-Image /RestoreHealth

Use this to understand how much of your WinSXS is actually shared with (actively used by) Windows, and how much is taken up by unused features / packages and other stuff:
dism.exe /Online /Cleanup-Image /AnalyzeComponentStore

Use this to get rid of the files from before a service pack was applied:
Dism /online /Cleanup-Image /SPSuperseded

And this one will make sure only the latest necessary version of each file in WinSXS is kept around:
Dism /online /Cleanup-Image /StartComponentCleanup /ResetBase

More aggressive cleansing

Should you want to get rid of all the unused features as well, you can do this:
DISM.exe /Online /English /Get-Features /Format:Table > features.lst

Then you can use this short bash command line to create a .bat file to remove all files that belong to disabled features: (use your Linux, or for windows: use the Linux subsystem in Windows 10, Cygwin on earlier versions of Windows to run bash)
grep '| Disabled\s*$' CMOD-features.lst|cut -d '|' -f 1|while read; do echo "DISM.exe /Online /Disable-Feature /featurename:${REPLY// /} /Remove"; done > remove-disabled-features.bat

Then just run the created .bat file to remove all the rest of the cruft. In theory Windows will be able to reinstall these files if you turn on any of the features, for which the files were removed, but I would not go ahead with the removal until I am reasonable sure I only remove what I will not need. YMMV!

For Windows 7 and Windows Server 2008 R2

It's always a good idea and can avoid a lot of strange error messages to make sure the Deployment Image is in a healthy condition:
dism /online /Cleanup-Image /scanhealth

Use this to get rid of the files which were deprecated by installed Service Packs:
DISM.exe /Image:C:\test\offline /Cleanup-Image /spsuperseded /hidesp

You can get the list of packages, and seems safe to remove the ones that are not "Installed" but "Staged" or "Superseded"

dism /online /Format:Table /Get-Packages > packages.lst

Then remove them, for example:
dism /online /remove-package /packagename:Microsoft-Windows-CodecPack-Basic-Package~31bf3856ad364e35~amd64~~6.1.7601.17514

You can also list the installed features both enabled and disabled:
dism /online /get-features >features.lst

But so far I have found no way on the older Windows versions to remove the files from WinSXS that belong to disabled features...

Extract all your Pictures, Videos, PDFs from iPhone backup!

I hate the fact that is so hard to retrieve all my files that I have on the iPhone (this is one of the reasons why I am switching from iPhone to an android device). They are all in the iPhone backup, but with their names scrambled, without extensions is impossible to sort thousands of files.

But not any longer, see how you can get access to all your files and data!
  1. Create a backup of your iPhone on your computer, that is _not_ encrypted (as described here)
  2. Find the files of your backup:
    • on Windows 8.x and 10 it is here: c:\Users\\AppData\Roaming\Apple Computer\MobileSync\Backup\
    • on Windows 7 and Vista: C:\Users\\AppData\Roaming\Apple Computer\MobileSync\Backup\
    • on Windows XP: C:\Documents and Settings\\Application Data\Apple Computer\MobileSync\Backup 
    • on OS X: ~/Library/Application Support/MobileSync/Backup/
  3. Each backup name is a 40 character long SHA (seems like garbage: 00d07a612db092c28c316244cce7d9199f23da33).
    Inside of them you will find another 256 folders named 00-ff, and inside those are your files, also with the long hash (garbage) names.
  4. I have created the below script, that will:
    - sort your files into target folders by type, and add their correct extensions
    - decode the Apple Binary Property Lists and SQLite databases, so that you can view and search them in clear text! (databases will also be copied there besides the dump if you want to open and query them!)
  5. You will need bash, sqlite3 and plistutil to run it, either on your Linux, or for windows: use the Linux subsystem in Windows 10, Cygwin on earlier versions of Windows.
  6. Please do not forget to replace the backup folder name and the target directory name with yours!
#!/bin/bash
#
# Created to compare speed and compression on a VM image
# to be run in the VM working directory e.g. /mnt/d/VMs
#

# Find the utils we will need
if ! which sqlite3 >/dev/null; then echo "Please install sqlite3 to dump database file contents!"; exit; fi
if ! which plistutil >/dev/null; then echo "Please install sqlite3 to dump database file contents!"; exit; fi

# Identify file types
# Put target directory here!

TDIR="/mnt/c/Users//Desktop/iPhone6.HU"
mkdir -p "$TDIR/Pictures/JPEGs/" "$TDIR/Pictures/PNGs/" "$TDIR/Text/" "$TDIR/XML/" "$TDIR/PDF/" "$TDIR/PlistToXML/" "$TDIR/Database/" "$TDIR/Audio/" "$TDIR/Contacts/" "$TDIR/Pictures/TIFFs/" "$TDIR/Fonts/" "$TDIR/Others/" "$TDIR/Movies/"
 

# Put your backup folder here!
file --no-pad /mnt/c/Users//AppData/Roaming/Apple\ Computer/MobileSync/Backup//*/* |

while read
do
  SRC="${REPLY%%:*}"
  FLNAME="${SRC##*/}"
  case "${REPLY#*: }" in
  (JPEG*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Pictures/JPEGs/$FLNAME.jpg"
    ;;
  (PNG*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Pictures/PNGs/$FLNAME.png"
    ;;
  (ASCII\ text*|*Unicode\ text*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Text/$FLNAME.txt"
    ;;
  (XML\ document\ text*)
    cp --preserve --no-clobber "$SRC" "$TDIR/XML/$FLNAME.xml"
    ;;
  (PDF*)
    cp --preserve --no-clobber "$SRC" "$TDIR/PDF/$FLNAME.pdf"
    ;;
  (Apple\ binary\ property*)
    plistutil -i "$SRC" -o "$TDIR/PlistToXML/$FLNAME.xml"
    ;;
  (SQLite\ 3*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Database/$FLNAME.sqlite"
    sqlite3 "$SRC" .dump >"$TDIR/Database/$FLNAME.sql"
    ;;
  (MPEG-4\ LOAS*|*AAC*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Audio/$FLNAME.aac"
    ;;
  (vCard*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Contacts/$FLNAME.vcard"
    ;;
  (TIFF*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Pictures/TIFFs/$FLNAME.tiff"
    ;;
  (TrueType*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Fonts/$FLNAME.ttf"
    ;;
  (*QuickTime\ movie*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Movies/$FLNAME.mov"
    ;;
  (*)
    cp --preserve --no-clobber "$SRC" "$TDIR/Others/$FLNAME"
  esac
done

 
Sample output folder:

Enjoy!

Should you add more interesting file types, please add them in the comments section!

How to transfer iPhone (SMS) messages to Excel

It's very easy to get your messages out of your iPhone:
  1. Create a backup of your iPhone on your computer, that is _not_ encrypted (as described here)
  2. Find the files of your backup:
    • on Windows 8.x and 10 it is here: c:\Users\\AppData\Roaming\Apple Computer\MobileSync\Backup\
    • on Windows 7 and Vista: C:\Users\\AppData\Roaming\Apple Computer\MobileSync\Backup\
    • on Windows XP: C:\Documents and Settings\\Application Data\Apple Computer\MobileSync\Backup 
    • on OS X: ~/Library/Application Support/MobileSync/Backup/
  3. Each backup name is a 40 character long SHA (seems like garbage: 00d07a612db092c28c316244cce7d9199f23da33).
    Inside of them you will find another 256 folders named 00-ff, and inside those are your files, also with the long hash (garbage) names.
  4. Go into the folder '3d', find the file 3d0d7e5fb2ce288813306e4d4636395e047a3d28
  5. Copy it for example to your desktop, rename it to: 3d0d7e5fb2ce288813306e4d4636395e047a3d28.sqlite
  6. Download the free and open-source SQLite database browser: http://sqlitebrowser.org/
  7. Install it, start it and open the .sqlite file from your desktop.
  8. Go to the Execute SQL tab, and enter:
    select  chat.last_addressed_handle,
            chat.chat_identifier,
            datetime(message.date + strftime('%s', '2001-01-01 00:00:00'),'unixepoch', 'localtime') as date,
            message.text,
            message.service
    from chat, chat_message_join,message
    where chat_message_join.chat_id=chat.ROWID and chat_message_join.message_id=message.ROWID
    order by last_addressed_handle,chat_identifier,date;
  9. Now you should get a table with 5 columns (from which phone, to which number, date, text and service) and as many rows as many messages you have saved in the backup!
  10. Save them as CSV:
  11. Now you can open that with Excel!
Extra step, if you have messages with international characters: you need to add three bytes (the UTF8-BOM) to the beginning of the file for Excel to recognize this:
  • You can either open the file with Notepad++ and save it as UTF-8-BOM as you see it in the screenshot

  • Or if you have bash, use this command line to add it:
    echo -ne "\xEF\xBB\xBF"|cat - iphone-messages.csv > iphone-messages-BOM.csv
 Enjoy!

Feel free to modify or extend the SQL to suit your needs, please post in the comments if you have a cool new one! 

2016. december 31., szombat

How to disable iPhone requiring backup encyption?

Your iPhone requires you to encrypt your backup?

... and you cannot find how to turn it off? 

iTunes surprised me with the message: "This iPhone requires backup files to be encrypted"

After a lot of digging I have found that this can be caused by installed Profiles (Crashlytics Provisioning Profile in my case, but some other people reported the same with an M1 (the telecommunication company) Profile installed).
To delete it:
  1. Go to Settings > General > Profile, tap on the app’s configuration profile.
  2. Then tap Delete Profile. If asked, enter your device pass-code, then tap Delete.
Then:
  1. Go to the Summary page for your device in iTunes
  2. double check "Automatically Back Up" is set to "This computer"
  3. double check "Encrypt iPhone backup" is _unchecked_
  4. Press "Back Up Now"
Congratulations, your backup will be in:
c:\Users\\AppData\Roaming\Apple Computer\MobileSync\Backup\

Now you can go ahead and get your text messages from it! :)

2016. december 29., csütörtök

VM image compression and optimization for transfer, distribution or deep-freeze

We often need to send and receive VM images, mostly over intercontinental and/or slow and/or high latency and/or unreliable connections. We had serious hardships in the past trying to receive last minute updates in such circumstances e.g. using hotel WiFi or 3G connections in Kuala Lumpur.

After some research it seems most important to clean up the inside of the image before transfer to avoid up to 10 hours (in my test example) spent transferring unnecessary clutter.

Most important findings, dos and do not dos follow, detailed instructions and data afterwards!

Most important findings

  1. You can often gain more size reduction from cleanup of the VM, than compression! In my example a 39M demo image became 25GB after a few cleanup steps.
  2. Transferring with rsync (or scp) can compress, _but_ that uses "gzip -6" which compresses my initial image to 18GB - my test image was compressed to 7.23GB with 7zip after cleanup.
  3. Translating that to transfer / download times: I tested on an intercontinental link (Singapore-EU) having the average speed of ~320kByte/sec (~2.5Mbit/s) at the time. The transfer time avoided is almost 10 hours! (16:08:00 for compressed rsync vs. 6:15:46 for clean-up plus 7zip) Transfer amount also matters a lot if you are distributing to several parties.

Don't:

  1.  Don't use zip or gzip to compress VMs!
  2. Don't over-optimize compression settings, cleanup matters more and the extra time will not buy you too much extra compression (see my insane 7zip settings tested below).
See below how - should you have questions, ask away in the comments section!

Step 1: Cleanup!

After some research (see benchmark data below if interested) it seems most important to clean up the inside of the image before transfer to avoid up to 10 hours spent transferring unnecessary clutter.
In our example: IntelliVectorDemo/ VMware product demo image:
Size (bytes)StepStep
38,852,268,020-Original image (vmware*.log and vprintproxy*.log already deleted)
37,231,468,532clean up inside
  1. Must have KB2852386 to remove files from WinSXS.
  2. Set StateFlags(DWORD32)=1 in all keys under (create if did not exist): HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\VolumeCaches
  3. Run Disk Cleanup for C: and any other drives from Properties
  4. Run:
    Dism /Online /Cleanup-Image /RestoreHealth
    Dism /online /Cleanup-Image /SPSuperseded
    Dism /online /Cleanup-Image /StartComponentCleanup /ResetBase
  5. Must reboot the VM for the WinSXS cleanup to happen. Do VMware "Clean Up Disks..." afterwards.
  6. If you got the guts, be more agressive by removing unused packages and features!
29,947,840,826clean up VM imageAfter all the cleanup _in_ the VM run the VMware disk cleanup _on_ it to shrink the disk image.
VMware / VM / Manage / Clean Up Disks…
28,932,666,690CCleanerRun Crap Cleaner and CC Enhancer inside VM.
27,528,290,245JBOSSUse Hdgraph or WinDirStat to find out what else is taking up a lot of space, maybe stale log files, App Server temp files, batch folders etc. Are hanging around and are not necessary.
In this case: JBOSS EAP cleanup + VM/Manage/Clean Up Disks…
25,160,484,978installsThink about the purpose of the VM. E.g. if it is not explicitly installation of software, remove the sw installation files from it.
Altogether 13691783042 bytes, more then 1/3 of the image was not necessary for the demo image!

Step 2: Initial transfer / distribution

There are two important parts to the network transfer itself:
  1. Initial transfer (upload or download) of the VM image: lot more efficient in a compressed form. It is very important to choose the right compression: forget Zip (DEFLATE), Gzip (DEFLATE), Bzip (Burrows-Wheeler):
    7z using LZMA(2) is the best both in compression and in compression ratio per wall time (taking its multi-threading ready design into account).
  2. Update image with changes: can be more efficient on the decompressed image using rsync to only transmit the changed blocks (no tests run to verify).
Both can be accomplished using rsnyc (to transfer just the *.7z file or to update the VM image directory):
Upload:
rsync -avvrhz --partial --stats --progress --inplace [folder or 7z file name] user@server:/mnt/rsync/VMs/
Download:
rsync -avvrhz --partial --stats --progress user@server:/mnt/rsync/VMs/[folder or 7z file name] .
 
Of course you should change "/mnt/rsync/VMs" to the folder you use on the server.
User needs to have SSH access to the server and you need to have rsync and ssh installed locally (for windows: use the Linux subsystem in Windows 10, Cygwin on earlier versions of Windows).

Step 3: Updating the image (if necessary) 

Update image with changes: can be more efficient on the decompressed image using rsync to only transmit the changed blocks (no tests run to verify).
  1. Upload the 7z
  2. then unzip on the server
  3. others download the 7z
  4. unzip locally
  5. then keep syncing the changes on the decompressed images on the server and locally

For this the same rsync command above can also be used.

Benchmark data

Compression IntelliVector
Demo.zip
size
ratio
CPU used Wall
time
Max mem. used
vmware*.log and vprintproxy*.log deleted 38,852,268,020 100.00%


gzip --fast 19,496,966,923 50.18% 88% 19:45.0 1.4MB
gzip -6 (this is what rsync compression uses) 18,625,975,551 47.94% 95% 31:49.4 1.4MB
gzip --best 18,572,524,862 47.80% 115% 56:14.1 1.4MB
7z a -t7z -m0=lzma2 -mx=3 -mfb=32 -md=1m -ms=on 16,460,348,353 42.37% 285% 0:49:30 61MB
7z a -t7z -m0=lzma2 -mx=5 -mfb=32 -md=16m -ms=on 14,732,928,437 37.92% 326% 1:38:24 587MB
7z a -t7z -m0=lzma2 -mx=9 -mfb=64 -md=64m -ms=on 14,079,148,699 36.24% 329% 2:04:54 2212MB
7z a -t7z -m0=lzma2 -mx=3 -mfb=32 -md=1m -ms=on 8,854,261,456 22.79% 312% 0:25:10 61MB
7z a -t7z -m0=lzma2 -mx=5 -mfb=32 -md=16m -ms=on 7,617,172,329 19.61% 284% 1:18:22 589MB
7z a -t7z -m0=lzma2 -mx=9 -mfb=64 -md=64m -ms=on 7,230,345,981 18.61% 283% 1:42:12 2070MB
7z a -t7z -m0=lzma2 -mx=9 -mfb=256 -md=256m -ms=on 7,107,098,190 18.29% 278 2:36:43 5102MB

Test environment


Environment
Processor i7-2640M @ 2.80GHz 2 cores+HT (4 virtual cores)
Disk Toshiba external 500GB USB3
RAM 8GB Physical
PC Lenovo X1 2011 (used since 2012 Feb 22)
OS Windows 10 64bit version 1607
linux Windows 10 Linux subsystem Ubuntu 14.04.5 LTS trusty
Vmware VMware Workstation Pro 12.5.2 build-4638234 Virtual Disk Manager
7zip 7-Zip [64] 9.20  p7zip Version 9.20 (HugeFiles=on,4 CPUs)
gzip gzip 1.6
test 1 CMOD95base 100GB+ image with several snapshots (8 vmdk files)
test 2 IntelliVectorDemo simple VMware player image (1 vmdk)

Additional interesting findings

  1. 16GB of the 25GB image is Windows (Server 2008 R2) and 8GB of that is the WinSXS folder. IT seems like a good idea to use a more recent version of Windows which also allows more flexible cleanup options with DISM.exe.
  2. Gzip can only use a single CPU core while 7zip used all of my cores in my test. So 7zip on a modern CPU is actually not so much slower!
  3. 7zip used 280%-320% CPU in my test of a 2 core +  Hyper threading CPU. Contrary to my previous beliefs HT actually adds significant bandwidth to the CPU!
  4. vmware-vdiskmanager.exe did not have any significant shrink or defrag effect on my .vmdk file, wonder why... The VMware Workstation GUI "Clean Up Disks..." worked though.

2016. május 30., hétfő

How to make BackBlaze faster?

I was struggling a lot to make uploads of BackBlaze complete faster. An have discovered an issue under Windows 10 (might be relevant on earlier versions as well) while the system is was heavy load: a process called MsMpEng.exe was using of a lot of disk and CPU: Windows Defender checking everything mindlessly.
I had added exclusions even for itself, and this made the system a lot more responsive and performant!

Process exclusions

I tried to add only processes which only read files, big files, or lots of them, but are not used to put new files onto the system (as opposed to e.g. totalcmd.exe)
I added these exclusions:
  • 7zG.exe
  • bztransmit64.exe
  • bztrans_thread00.exe
  • bztrans_thread01.exe
  • bztrans_thread02.exe
  • bztrans_thread03.exe
  • bztrans_thread04.exe
  • bztrans_thread05.exe
  • bztrans_thread06.exe
  • bztrans_thread07.exe
  • bztrans_thread08.exe
  • bztrans_thread09.exe
  • dwm.exe
  • deluge.exe
  • google photo backup.exe
  • kodi.exe
  • msmpeng.exe
  • perfmon.exe
  • searchfilterhost.exe
  • searchindexer.exe
  • searchprotocolhost.exe

File type exclusions

Mostly large, non-executable media files:
  • avi
  • jpeg
  • jpg
  • mkv
  • mp3
  • mp4
  • mpeg
  • mpg
  • orf
  • ts

Location exclusions

  • C:\Program  Files\Windows Defender
  • ... 

2015. december 12., szombat

Debunking wifi speed myths - What is really best for you?

I did not pay attention to the advances in wifi technologies since I bought my - at the time - best performing Wireless-N router and have been happily using that with my old but trusty Lenovo X1.

N300, N450, N600, N900, AC1750, AC1900, AC2350, AC3200, AC5300 etc. how much better are all the shiny new routers?

This started out as an my research into:
  1. What all those ACxxxx numbers mean?
  2. How much real word benefits do they really deliver? 
  3. How should I choose the best  router for my not-so-busy home network?
In the first three sections I summarize why that is misleading / worthless / not relevant in themselves. And show you how fast these are in practice.

Basic math, right?

If a:
  • class N300 Wireless-N device has a 300 Megabit/second maximum link rate...
Will you be able to connect to your new shiny (and expensive)
  • class AC5300 Wireless-AC router with 5300Mbps (5.3Gigabit per sec)?
No! There is increasingly subtle difference between the theoretical numbers used by marketers and the real world measurement!

Quick answers in the next section!

So how should I choose?

I came up with these simple rules in the end:
  1. All routers lately offer very solid WAN-to-LAN routing performance - no real difference in that unless you will have a connection faster then 500Mbps to the internet.
  2. I consider  non essential features like storage sharing on an USB port irrelevant since I have dedicated solutions for these.
  3. Only choose triple radio devices if you really have several (more than 15-20) devices _and_ there are not a lot of wifis around already. Otherwise one 2.4 and one 5.0 GHz radios are enough.
  4. It's no use buying any device advertising NitroQAM or 1024-QAM (for this) since there are no client devices implementing this non standard modulation.
  5. However buying devices which offer the TurboQAM meaning 256-QAM on 2.4 GHz might make sense. It's non-standard on 2.4, but standard on 5.0 so I expect client device manufacturers that already have it for 5.0 will also enable it for 2.4.
  6. MU-MIMO does not really help you till the next 2-3 years, till compatible devices replace your current phones, tablets and laptops.
  7. Having several antennas and streams will not help the maximum connection speed of your device, until it is only capable of one or two streams.
  8. You should focus on the Downlink speeds: uplink is usually similar and you will be using downlink 5x-10x more (and will probably never saturate both directions simultaneously).
  9. If you have a small or medium flat, you probably will have all of it in range in both 2.4 and 5.0 GHz, so I would suggest you find the router with the best 5.0 GHz speed at high distances
    See here: http://www.smallnetbuilder.com/tools/charts/router/bar/119-5-ghz-profile-dn?see=P_42 - select 42 dB attenuation (distance and/or walls): none of the routers available at the time of writing this article can connect at the maximum 45 dB attenuation (and keep the connection from any direction around the router)
    - filter for only the most realistic test method Revision 8 - only the magenta bars (the earler ones are less real world like)
  10. If you have a large apartment or house to cover with a single router/AP you probably will not have 5.0 GHz coverage in all of it, but at least you should have the best 2.4 GHz even in the farthest corner. Look for the router with the best 2.4 GHz speed at high distances!- See here: http://www.smallnetbuilder.com/tools/charts/router/bar/117-2_4-ghz-profile-dn?see=P_63 - select the highest 63 dB 63db attenuation (distance and/or walls)
    - filter for only the most realistic test method Revision 8 - only the magenta bars (the earler ones are less real world like)
    - there are several routers that will give you connection, even acceptable (10Mbps+) speed even in the worst circumstances! (remember, the best speed of Wireless-B 802.11b was only 11 Mbps :)
It still makes sense to buy Wireless-AC routers/APs, they also improve range and speed for your old Wireless-N devices, probably because of using more advanced electronics and better quality radio components.

Should you want to understand my research and reasoning behind, below are all the gory details! :)

It really not the largest number that matters!

Let's start by clarifying what kind of maximums numbers we have. Below I show you how much is the difference between the standards maximum speeds vs. the marketed router maximum aggregate speeds vs. the real word usage maximum speeds.
  •  my decent but very old setup: NetGear WNDR3700 N600 router from 2009
    with my 2011 Lenovo X1 laptop (Intel Centrino Advanced-N 6205 2x2 adapter)
  • NetGear AP on 5GHz with the fastest smartphones on market in 2015:
    Samsung Galaxy S6 and S6 Edge, iPhone 6 and 6 Plus

Speed of what? Wireless-N Wireless-AC
Theoretical maximum throughput of the standard 600 + 600 Mbps (on 2.4 and 5 GHz at the same time)6934 Mbps
Theoretical maximum aggregate throughput of the access point 300 + 300 Mbps (on 2.4 and 5 GHz at the same time)450 + 1300 Mbps
Theoretical maximum throughput of the best wireless network of the Access Point 300 Mbps1300 Mbps
Maximum link rate of a device to a wireless network 300 Mbps433 Mbps
Maximum measured combined throughput (up and down at the same time) 183 Mbps355 Mbps
This is how you may actually use it!Maximum measured download throughput (up and down at the same time) 92 Mbps186 Mbps

As you can see, even if the standards in theory are very advanced the actual connection and transfer speed is not very different, not more than what's expected with 4-6 years between the setups:


So unless you want to serve wifi to several high performance devices at your home you should not be looking for a Router / Access Point with the highest classes and numbers but the ones that are tested to give your devices decent speed with high coverage (in the farthest corner of your flat)!
See a good explanation of wifi terminology here.

How fast are the Wireless-N and Wireless-AC classes?

If you want to know, what YOU will get, considering your device:
  • 2x2 old, Wireless-N laptop and router: 300 Mbps link rate and 92 Mbps actual download speed
Wireless-AC device:
  • 1x1 smartphone: max. 433 Mbps link rate, less than 200 Mbps actual download speed
  • 2x2 laptop adapters: max. 866 Mbps link rate, less than 250 Mbps actual download speed
(Even the late 2015 models are not faster: Lenovo Carbon X1 with Intel Wireless-AC 7265 adapter,
Microsoft Surface Pro 4 with the Marvell Avastar 88W8897 wireless chip, latest Macbooks seem to have a unique BCM15700A2 chip, also 2x2 AC)
As you can see there is not a huge difference between an ages old N600 router and a brand new AC5300 class one!

Still if you have multiple high-traffic devices, you might benefit from Wireless-AC routers that has 2 or 3 radios (offers 2-3 separate wireless networks) and four streams at the same time (for serious bucks!). But please be aware! All those antennas and radios only talk to one device at a time (SU-MIMO) even if you buy a fancy 4 antenna/stream router, your laptop only has max. two!
There is hardly any MU-MIMO router and the market and really no MU-MIMO laptops and smartphones. Maybe in 2016. Yeah, and did I tell you even if a MU-MIMO router will be able to talk to multiple devices at the same time on its multiple antennas, it only works router to device and not the other way around.

Meaning of Wireless class codes

So let's see, what the numbers mean:
N150: a single wifi, maximum 150Mbps (1 stream, 64-QAM, 40 MHz channe, 400ns GI:)
N300: a single wifi, maximum 300Mbps (same with 2 streams)
N450: a single wifi, maximum 450Mbps (same with 3 streams)
N600: maximum 300 Mbps!
  • even a single wifi could be this fast in theory with , but in practice no 4x4 devices exist to connect to a 4x4 AP, also the even 4x4 antenna APs mostly only handle 3 spatial streams.
  • so in reality, AccessPoints marketed as N600 are actually offering two 300 Mbps Wireless-N wifis (one 2.4 GHz and a 5 GHz) - either one you connect to, you will only get 300 Mbps.
N750: maximum 450 Mbps! Similarly this only means one 300 and a 450 Mbps wifi.
N900: maximum 450 Mbps! Similarly this only means 2 x 450 Mbps wifis.


Router class 2.4 GHz radio 5 GHz radio 2nd 5 GHz radio Aggregate
Best channel modulation Mbps Streams modulation Mbps Streams modulation Mbps Streams Mbps
in the standard 64-QAM 600 4 x 150 256-QAM 3 464 8 x 433 256-QAM 3 464 8 x 433 7 528
AC5300 1024-QAM 1 000 4 x 250 1024-QAM 2 166 4 x 542 1024-QAM 2 166 4 x 542 5 333
AC3200 256-QAM 600 3 x 200 256-QAM 1 299 3 x 433 256-QAM 1 299 3 x 433 3 198
AC3100 1024-QAM 1 000 4 x 250 1024-QAM 2 166 4 x 542




3 166
AC2600 256-QAM 800 4 x 200 256-QAM 1 732 4 x 433




2 532
AC2400 256-QAM 600 3 x 200 256-QAM 1 732 4 x 433




2 332
AC2350 64-QAM 600 4 x 150 256-QAM 1 732 4 x 433




2 332
AC1900 256-QAM 600 3 x 200 256-QAM 1 299 3 x 433




1 899
AC1750 64-QAM 450 3 x 150 256-QAM 1 299 3 x 433




1 749
AC1600 64-QAM 300 2 x 150 256-QAM 1 299 3 x 433




1 599
AC1300 64-QAM 450 3 x 150 256-QAM 866 2 x 433




1 316
AC1200 64-QAM 300 2 x 150 256-QAM 866 2 x 433




1 166
AC750 64-QAM 300 2 x 150 256-QAM 433 1 x 433




733
AC580 64-QAM 150 1 x 150 256-QAM 433 1 x 433




583
N900 64-QAM 450 3 x 150 64-QAM 450 3 x 150




900
N750 64-QAM 450 3 x 150 64-QAM 300 2 x 150




750
N750 64-QAM 300 2 x 150 64-QAM 450 3 x 150




750
N600 64-QAM 300 2 x 150 64-QAM 300 2 x 150




600
N450 64-QAM 450 3 x 150









450
N300 64-QAM 300 2 x 150









300
N150 64-QAM 150 1 x 150









150
G54 64-QAM 54 1 x 54









54


How do they make wifi get faster and faster?

Wifi creators can increase speed (throughput) in at least three major ways:
  1. Using more sophisticated radio modulations: e.g. 256-QAM (marketing calls it TurboQAM) of Wireless-AC can cram 200 Megabits/s into the same stream example 64-QAM of Wireless-N does 150 Mbps - like if you were sending natural gas through tubes: if you compress them more you can send more in same time through the same tube (in theory: in a noise-less environment, which is usually not the case). This only needs smarter radio design, computing and signal processing (chip with maybe a bit higher power consumption), but no significant physical expenses.
  2. Using wider radio bands (wider tubes) started with 20 Mhz bands, now Wireless-AC allows to use four and eight times wider bands (Wifi devices do not work exactly at 2400 MHz and 5000 MHz but a few of such channels above those.) This still only needs just frequency band allocation (and maybe more power).
  3. If that is still not enough it's easier to just aggregate multiple streams of data, like using 2, 3 or 4 of these stuffed and fat pipes. This on the other hand needs multiple antennas and those also need to be driven with radio power, meaning more physical parts taking up space and ultimately higher costs!This is also called MIMO, SU-MIMO when you Wifi Access Point (AP) can use that many pipes but still only talk to one device at a time, MU-MIMO when it can use some of the pipes to talk to your smartphone and some others to your laptop at the same time (still only one can talk back at a time though).

    +1: For 11% increase in throughput you can have a 400 nanosecond Guard Interval (GI) instead of 800 ns, this is SGI (Short Guard Interval), not all routers support it. It will not help if there is high interference, since GI is there to protect against interference.

Can I multiply speed using several streams?

In theory, yes!
This is usually indicated as 2x2 or 3x3 (for 2 or 3 receive and transmit antennas). Some devices are 1x2 transmiting on 1 but receiving on 2 since downloading data is more common than sending, especially on smartphones and tablets
This is also to say 4x4 client devices are extremely rare to non-existent.

In practice even 3x3 is rare for Wireless-AC currently: Intel wireless adapters (commonly built into laptops and ultrabooks) are available either in:
  • 3x3 Wireless-N or
  • 2x2 Wireless-AC.

Why the devices often cannot use the fastest speed the AP offers?

Wifi speed depends on the device using it as well:
  1. Your smartphones typically have only one antennas and can handle only one or two of the above pipes (streams). This is because of space constraints, battery usage, and frankly you just could not use it even if you had higher bandwidth. :)
    - iPhone 4 was max. 20 MHz channel, 64-QAM, SGI and 1 stream only, thus max. 72.2Mbps
    - iPhone 5 can do 40 MHz, so its best is 150 Mbps but still 1x1:1.
    - iPhone 6 is still limited to a single stream, but it does Wireless-AC:
      - 433Mbps on 5 GHz using 80 MHz channel, 256-QAM and SGI.
      - 200Mbps on 2.4 GHz with 256-QAM and SGI, limited max. 40 MHz channels.
    - Samsung Galaxy S5, S6 and Note 5 has a 2x2.
  2. Tablets usually handle 2 streams.
  3. Laptops with Wireless-N Adapters most handle at least 2 streams since a long time, sometimes 3.

How wide channels can get?

Okay, if my devices cannot have several streams (and antennas) can I use the 80 and 160 MHz channels of Wireless-AC?
In theory, yes!
But in practice you are limited:
  • 2.4 GHz can only use 20 and 40 MHz channels (both mandatory in both Wireless-N and AC)
  • 5 GHz can use 80 (mandatory in Wireless-AC) and 160 MHz (optional, not all APs and adapters do it!) channels, but they are not feasible in places where there may be several other wifis (like public spaces or apartment buildings).
Why? To understand we have to look into how original 20 MHz channels are mapped around 2.4 GHz. In the US channels 1-11 are available, in Europe channels 1-13. However this does not mean, that we have e.g. 13 x 20 MHz wide band available for wifi at 2.4 GHz. Below is the inSSIDer map of my (40 MHz wide) wifi (indicated with blue lines) in our apartment building - several other wifis also exist and do overlap (shown with red).


The channels are laid out very much overlapping:
  • every 5 MHz there is the center of a 20 MHz band, 10 MHz left of the center and 10 MHz righ
  • So in reality the total frequency space allocated for Wifi at 2.4 GHz only allows for three non-overlapping wifi channels (that are not even adjacent on the edges: protocol requires 16.25 to 22 MHz of channel separation): 1-6-11 (or, if you are not in the use you may also choose 2-7-12 or 3-8-13)
  • As you can see even here there is only space for 2 non-overlapping 40 MHz channels (and they will still be adjacent) if you are not in the US.
  • In the US, out of luck: you can only have one non-overlapping 40 MHz channel at a time!
  • Also even if you started 10 MHz left of channel 1 and finished 10 MHz right of channel 13 (that is only allowed outside of the US) that would only give you a single 80 MHz channel - that is why 80 and 160 MHz wide channels are not allowed at 2.4 GHz!

5 GHz wifi has more frequency space

For comparision here is an inSSIDer map of the 5 GHz channels in my building:

It's immediately obvious:
  • 5 GHz is less crowded, since it's not provided by many APs (my Cisco device provided by the carrier included). This is good!
  • 5 GHz is less crowded since is's signal weakens faster, so my laptop only sees APs really close to my flat. This is good because all the rest of the 5 GHz wifis will not interfere with mine!
  • 5GHz has a lot more _and_ wider channels allocated: the 36+40 channel shown has 40 MHz width taking up some one tenth of the total space - same 40 MHz width in 2.4 Ghz takes up exactly half of the total space!
  • As you can maybe  guess from varied the channel spacing in the above map, 5 GHz has channels already defined with 80 and even 160 MHz width that are actually just aggregations of narrower channels shown in the below table. (Details: See them all here)
20 MHz40 MHz80 MHz160 MHz
36} 38} 42} 50
40
44} 46
48
52} 54} 58
56
60} 62
64
100} 102} 106} 114
104
108} 110
112
116} 118} 122
120
124} 126
128
132} 134} 138
136
140} 142
144
149} 151} 155
153
157} 159
161
165

Summary

Wireless-N (in theory) can have a maximum of 4x4 antennas, 4 streams and 20 or 40 MHz channels. Highest data rate modulation is 64-QAM.
Wireless-AC (in theory) can have a maximum of 8x8 antennas, 8 streams and 20, 40, 80 and 160 MHz channels Highest data rate modulation is 256-QAM (1024-QAM is not standard but supported by some devices).
But in practice most of the higher speed features of AC do not work at 2.4 GHz, and the super wide channels  that could be used at 5 GHz will also suffer if there is any other network (e.g. public spaces, apartment buildings).

How to tell which of those tweaks your wireless device uses from the link rate?

 If you already have a Wireless-N AP use your laptop, since it will tell you more, and probably is capable of more streams than any other of your devices, go really close to the AP . If you get a:
  • 130Mbps connection: you are using 2 streams, 64-QAM, normal GI (800ns), 20 MHz channel
  • 144Mbps connection: you are using 2 streams, 64-QAM, SGI (400ns), 20 MHz channel
  • 270Mbps connection: you are using 2 streams, 64-QAM, normal GI (800ns), 40 MHz channel
  • 300Mbps connection: you are using 2 streams, 64-QAM, SGI (400ns), 40 MHz channel
Diagnose what parameters your best link rate from this Wireless-N table. Same table for Wireless-AC is here.
This is actually how I noticed my laptop never uses 40 MHz channels. So I went into Control Panel / Network and Internet / Network Connections, opened the Properties of my wireless adapters and found on the Advanced tab, that 802.11n Channel Width for band 2.4 GHz was set to 20 MHz only instead of Auto. :)

Does An AC Router Improve N Device Performance?

It is interesting to see, that even if you only own Wireless-N devices, having a state-of-the-art AC router will definitely get you significantly (2x, 3x, 4x!) better speeds in real life scenarios.
It cannot improve the _maximum_ speed of your router and laptop, but that only works if you sit _really_ close (1-2 meters, no wall) to the Wifi access point. In fact it's better to say it improves wifi speed at range: if you happen to be in another room or especially in the far edge of your apartment, a the best Wireless-AC routers will give you a lot faster speed on both 2.4GHz and 5GHz for Wireless-N devices as well!

See the detailed test and what routers perform well at one of my favourite AP, router, and NAS test site: http://www.smallnetbuilder.com/wireless/wireless-features/32512-does-an-ac-router-improve-n-device-performance.

Rendszeres olvasók