Guide: Getting Xilinx ISE to work with Windows 8 / Windows 10 (64-bit)

[robca]

PSA: There seem to be two new issues in newer versions of W10.

During installation, it hangs around 92% when checking Webtalk. What worked for me, was to terminate webtalk32.dll (under the ISE setup app in Task Manager), wait a moment, then terminate webtalk.dll. Setup ends seemingly successfully at that point

And ISE is broken again in Windows 10, but luckily someone smarter than me figured out is was due to an interaction with the "fake" WSL network shares causing problems to the network shares enumeration. The fix is either to remove WSL (not an option for me) or patch a file

https://www.exxosforum.co.uk/forum/viewtopic.php?p=95891&sid=7526ed0d6748614255e95d07bf38ce0b#p95891

User czietz patched libPortabilityNOSH.dll to always fail network shares enumeration and provided a fixed file (renamed to libPortability.dll, to solve the Smartheap problem already mentioned). I copied the patched files into both locations C:\Xilinx\14.7\ISE_DS\ISE\lib\nt64 and C:\Xilinx\14.7\ISE_DS\common\lib\nt64 and everything seems to work fine

Downloading a binary from a random forum is usually risky, but the patch is limited to a few bytes

0001AB30: BB 8B
0001AB31: 60 F8
0001AB32: 09 BB
0001AB33: 00 60
0001AB34: 00 09
0001AB35: 48 00
0001AB36: 8B 00
0001AB37: CB 48
0001AB39: F9 CB

0:  8b f8                   mov    edi,eax
2:  bb 60 09 00 00          mov    ebx,0x960
7:  48 cb                   rex.W retf

And if I interpreted the binary correctly, it simply returns early from a call, so pretty safe

Copying the file here, just in case. Hope it's ok with the original author

[suudy]

Sorry to resurrect this old thread.  We are working through the forced Windows 11 installs we are getting on new PC's at work now, and I've found this while digging on how to get ISE to work.

Downloading a binary from a random forum is usually risky, but the patch is limited to a few bytes

This does appear to work.  But I'm not sure about the explanation below.

0001AB30: BB 8B
0001AB31: 60 F8
0001AB32: 09 BB
0001AB33: 00 60
0001AB34: 00 09
0001AB35: 48 00
0001AB36: 8B 00
0001AB37: CB 48
0001AB39: F9 CB

0:  8b f8                   mov    edi,eax
2:  bb 60 09 00 00          mov    ebx,0x960
7:  48 cb                   rex.W retf

And if I interpreted the binary correctly, it simply returns early from a call, so pretty safe

The above disassembly doesn't seem quite right.

First, here's the disassembly, based on defuse.ca's online disassembler, of the original libPortabilityNOSH.dll at offset 0x1ab30:

0:  8b f8                   mov    edi,eax
2:  bb 60 09 00 00          mov    ebx,0x960
7:  48 8b cb                mov    rcx,rbx
a:  e8 45 04 05 00          call   0x50454

And here's the disassembly based on the attached DLL file:

0:  bb 60 09 00 00          mov    ebx,0x960
5:  48 8b cb                mov    rcx,rbx
8:  8b f9                   mov    edi,ecx
a:  e8 45 04 05 00          call   0x50454
f:  48 8b f0                mov    rsi,rax

Note that the call is a rel32 (32-bit offset).  But the point here is that it doesn't return from the function early.  It only changes the registers before calling whatever is at 0x50454.

The only difference here is that edi gets loaded with edx, rather than eax.  Assuming the Microsoft x64 calling convention (this is a 64-bit DLL), the arguments are pushed RCX, RDX, R8, R9.  RAX is non-volatile and is whatever is in the caller's state.

I'm not sure what this does, or how it does it.  But it doesn't appear to return early.

[suudy]

Ok, I've done some work and have a new version of the DLL that I think makes sense.  And importantly, *it works!*

I ran the existing libportabilityNOSH.dll through Ghidra and decompiled things.  The previous patch appears to bypass the code after a call to WNetOpenEnumA by manipulating the return value.  This isn't clean, and seems to mess with arguments to the function immediately afterward (call to "new").

So, where the original patch did it's work was at file offset 0x1ab30.  Based on the DLL headers, this maps to a load address offset of 0x1000, and the .text section starts at file offset 0x400.  This means the address of the patched code is at a virtual offset of 0x1b730.  A disassembly/decompiling of the code yields:

18001b729 8b ca           MOV        ECX,EDX
18001b72b e8 2a 04      CALL       MPR.DLL::WNetOpenEnumA                           DWORD WNetOpenEnumA(DWORD dwScop
                 05 00
18001b730 8b f8           MOV       EDI,EAX
18001b732 bb 60 09      MOV        EBX,0x960
                 00 00
18001b737 48 8b cb      MOV        RCX,RBX
18001b73a e8 45 04      CALL       MSVCR90.DLL::operator_new                        void * operator_new(__uint64 par
                 05 00

Note that the patch modified the code *after* the call to WNetOpenEnumA.  But it did so in an odd way.  The return value from WNetOpenEnumA is in EAX.  The original code saves this in EDI, then sets up a call to "new".  Here's what original patch does:

18001b729 8b ca           MOV        ECX,EDX
18001b72b e8 2a 04      CALL       MPR.DLL::WNetOpenEnumA                           DWORD WNetOpenEnumA(DWORD dwScop
                 05 00
18001b730 bb 60 09      MOV        EBX,0x960
                 00 00
18001b735 48 8b cb      MOV        RCX,RBX
18001b738 8b f9           MOV        EDI,ECX
18001b73a e8 45 04      CALL       MSVCR90.DLL::operator_new                        void * operator_new(__uint64 par
                 05 00

This effectively only tweaks the return value of WNetOpenEnumA.  That is, it fails to store the return value of WNetOpenEnumA into the local stack frame, leading to the later checks to fail, bypassing much of the code.  This works, but does not completely bypass the function call.

Instead, I have proposed the following 4 byte patch which does what was suggested in the earlier email.  This patch returns earlier from the function, and fixes the checksum of the DLL.  Here's the change:

18001b700 c3              RET
18001b701 90              NOP

That's it!  And it works.

So, there are only 4 bytes in the entire file to change:

00000151 D1 54
00000152 0D 4B
0001AB01 40 C3
0001AB02 53 90

Attached is a bsdiff patch file.  See this link for details on how to apply it to libPortabilityNOSH.dll.

 libPortabilityNOSH-patch.zip (0.34 kB - downloaded 456 times.)

[auxym]

Thanks @suudy for that patch! Is it for the 32 bit (in the nt folder) or the 64 bit (in the nt64) version of libPortability?

Edit: reason is ask is because I am using LabVIEW FPGA, which appears to be loading the 32 bit version (based on what Process Explorer tells me), even on W11.

[glenenglish]

For using 23 year old FPGA devices,
I would suggest run something like Windows XP  in a virtual machine, and run ISE on that.

[iMo]

FYI - I tried with this copy batch (it is not a patch, it only copies the existing ISE14.7 files from A to B).

https://github.com/cbureriu/xilinx-14.7-patch-for-Win10-32-64/tree/master

and I can build my old Win7 s6 and v6 projects in Win10 no problem (both ise and planahead).

[SiliconWizard]

Works fine on Linux otherwise.

[auxym]

I work in academia, I asked for a win 7 VM and they won't even allow that, security concerns and everything

(Don't tell them I'm executing patched DLLs downloaded from internet forums)

[SiliconWizard]

I work in academia, I asked for a win 7 VM and they won't even allow that, security concerns and everything

Damn. Tell them that it's all ok because you're going to use Crowdstrike.

[ali_asadzadeh]

Can we run this damn old tool on win 11 too? since many customers still use spartan 6 and I need the ISE

[ali_asadzadeh]

Hi,
I want to know if it's possible to install ISE14.7 on win 11 using windows WSL? is it possible? has anyone tried it, since Today I  heard about WSL and I'm a total noob.

[robca]

Hi,
I want to know if it's possible to install ISE14.7 on win 11 using windows WSL? is it possible? has anyone tried it, since Today I  heard about WSL and I'm a total noob.

ISE is a Windows app. WSL is used to run Linux apps under Windows, so completely useless in this case

ISE 14.7 works in Windows 10, as long as you follow the information here and use the modified DLLs

[Someone]

Hi,
I want to know if it's possible to install ISE14.7 on win 11 using windows WSL? is it possible? has anyone tried it, since Today I  heard about WSL and I'm a total noob.

ISE is a Windows app. WSL is used to run Linux apps under Windows, so completely useless in this case

ISE is also available as a native linux app (suite would be a better term), but was targeted at specific linux versions so can be a bit a work to get it running on something newer.

[ali_asadzadeh]

ISE can be used on Linux too, But I do not know if there are limitations on that too, has any one have experience on WSL or Linux installation, so can guide us to try it, if it can be done, it would solve a huge problem.

[robca]

ISE can be used on Linux too, But I do not know if there are limitations on that too, has any one have experience on WSL or Linux installation, so can guide us to try it, if it can be done, it would solve a huge problem.

It would not solve any "huge problem", mostly because there are no huge problems in the Windows version. 

On the other hand, the Linux version requires quite a few steps to install, plus a few manual fixes https://gist.github.com/aliemo/ce58ea570ee6ffa6dedfa569f87f1c1e?permalink_comment_id=5040735. That even before figuring out if you properly configured WSL for graphical operations, and the necessary hardware drivers (which are always a problem under WSL emulation)

If you have a Windows machine, use the Windows version and follow the easy fixes described here to make it work. It works just fine

[SiliconWizard]

The dynamic libraries fixes are annoying but pretty common for closed-source Linux software. I have to do it also for Lattice Diamond & Radiant on Linux. Once that's done, the tools works fine, exactly as on Windows (for me at least). The only thing is that GUI font size may be a bit too small or too large, it'll rarely match that of your desktop environment, small annoyances like this. But that's minor.

[ali_asadzadeh]

If you have a Windows machine, use the Windows version and follow the easy fixes described here to make it work. It works just fine

Is there any workaround win 11? I could not install it on win 11

[robca]

If you describe the steps you tried and the errors you get, someone might be able to help. Windows 11 is pretty much just like Windows 10 for the most part, so the Windows 10 workarounds should work on 11 as well

[ali_asadzadeh]

I tried to install it on win 11 and During installation, it hangs around 92% when checking Webtalk.
Then I tried to end webtalk32 in task manager, like this hint by the useres

What worked for me, was to terminate webtalk32.dll (under the ISE setup app in Task Manager), wait a moment, then terminate webtalk.dll. Setup ends seemingly successfully at that point

But it hangs again and it would not go any further.

[robca]

I tried to install it on win 11 and During installation, it hangs around 92% when checking Webtalk.
Then I tried to end webtalk32 in task manager, like this hint by the useres

What worked for me, was to terminate webtalk32.dll (under the ISE setup app in Task Manager), wait a moment, then terminate webtalk.dll. Setup ends seemingly successfully at that point

But it hangs again and it would not go any further.

Yep, that was my post 

Did you terminate both webtalk32.dll, then after a few seconds terminate webtalk.dll?

Where does it hang for you? 92% or later?

[robca]

I tried to install it on win 11 and During installation, it hangs around 92% when checking Webtalk.
Then I tried to end webtalk32 in task manager, like this hint by the useres

What worked for me, was to terminate webtalk32.dll (under the ISE setup app in Task Manager), wait a moment, then terminate webtalk.dll. Setup ends seemingly successfully at that point

But it hangs again and it would not go any further.

Yep, that was my post 

Did you terminate both webtalk32.dll, then after a few seconds terminate webtalk.dll?

Where does it hang for you? 92% or later?

Just for my own knowledge, I downloaded the standard ISE 14.7 ISE for Windows 7/XP/Server Full Installer on my Windows 11 machine. After decompressing it, I launched setup as Administrator, and as per before when it stopped responding I killed webtalk32 and webtalk.exe (possibly even xwebtalk.exe) with Task Manager. After killing all instances of webtalk, the installation continued (VCredist, drivers) and got stuck again, this time on xilperl. Deleting xilperl completes the installation, but it's unclear if it works after that (I didn't have  my Spartan board to test it)

So, it looks as if Win 11 is causing more problems than Windows 10.

You can try this https://gist.github.com/islandcontroller/80f701cb84003408dfc3beecbc5207c1?OkHoR=ezTCjAb but, probably, the best approach is to use the ISE 14.7 Win10 edition, which is simply a VM with Windows 7 and preinstalled ISE, running in a virtual environment in Windows 11

[ali_asadzadeh]

Just for my own knowledge, I downloaded the standard ISE 14.7 ISE for Windows 7/XP/Server Full Installer on my Windows 11 machine. After decompressing it, I launched setup as Administrator, and as per before when it stopped responding I killed webtalk32 and webtalk.exe (possibly even xwebtalk.exe) with Task Manager. After killing all instances of webtalk, the installation continued (VCredist, drivers) and got stuck again, this time on xilperl. Deleting xilperl completes the installation, but it's unclear if it works after that (I didn't have  my Spartan board to test it)

So, it looks as if Win 11 is causing more problems than Windows 10.

You can try this https://gist.github.com/islandcontroller/80f701cb84003408dfc3beecbc5207c1?OkHoR=ezTCjAb but, probably, the best approach is to use the ISE 14.7 Win10 edition, which is simply a VM with Windows 7 and preinstalled ISE, running in a virtual environment in Windows 11

Another method would be asking linux and WSL experts in here how to do it in WSL in win11, since running a virtual machine has it's own limitation and crashes too!

[robca]

Just for my own knowledge, I downloaded the standard ISE 14.7 ISE for Windows 7/XP/Server Full Installer on my Windows 11 machine. After decompressing it, I launched setup as Administrator, and as per before when it stopped responding I killed webtalk32 and webtalk.exe (possibly even xwebtalk.exe) with Task Manager. After killing all instances of webtalk, the installation continued (VCredist, drivers) and got stuck again, this time on xilperl. Deleting xilperl completes the installation, but it's unclear if it works after that (I didn't have  my Spartan board to test it)

So, it looks as if Win 11 is causing more problems than Windows 10.

You can try this https://gist.github.com/islandcontroller/80f701cb84003408dfc3beecbc5207c1?OkHoR=ezTCjAb but, probably, the best approach is to use the ISE 14.7 Win10 edition, which is simply a VM with Windows 7 and preinstalled ISE, running in a virtual environment in Windows 11

Another method would be asking linux and WSL experts in here how to do it in WSL in win11, since running a virtual machine has it's own limitation and crashes too!

WSL is a virtual machine, too (lightweight, but still a VM). VMs are used everywhere in production, including servers running the Internet infrastructure. AMD supports the VM installation of ISE 14.7, but does not support ISE 14.7 on Windows or WSL

So, if you want a reliable, supported option, use what AMD recommends: the VM install