[Suggest] NONMEM adds UTF-8 support for csv files

From: [email protected] <[email protected]> On Behalf Of HUI, Ka Ho Sent: Monday, June 4, 2018 4:48 AM To: Mark Tepeck <[email protected]>; [email protected] Subject: [FORGED] [NMusers] 回覆: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Dear Mark Does it work if you don't put "ID" as the first column? Back to the days when I frequently used Excel, this had been the solution for me. Matthew ------ 原有訊息------ 寄件者： Mark Tepeck 日期: 2018年6月4日週一 10:41 收件者： [email protected]<mailto:[email protected]>; 副本： 主旨：[NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi NONMEM Users, As a PK scientist, I often manipulate csv files with Microsoft Excel, R, and NONMEM. As some of you may have seen, if the first record in a csv file is a string “ID”, Microsoft Excel would mistake it as an SYLK file which cannot be opened normally by Excel. One workaround is to use any strings other than ID, e.g. id, '‘ID’. However, this may cause new problems in some scenarios. Another potential solution is to use the R function readr::write_excel_csv ('dataframe.csv') to output a csv file with UTF-8 Byte order mark. Unfortunately, this csv file turned out to become unreadable for NONMEM although Excel likes it. Since waiting for bug fixing of Excel by Microsoft is not optimistic, I would truly appreciate it if NONMEM can add UTF-8 support. Thank you, Mark Tepeck

On Mon, Jun 4, 2018 at 7:52 AM, Mark Tepeck <[email protected]> wrote: > Hi All, > > > Unfortunately, the latest release of MS Excel 2017 (build 9330.2087 for > office 365) becomes more restrictive on SYLK files, which is apparently a > step-back update, but really a typical Microsoft behavior ☹. Even you > agree to continue with the warning messages, the latest Excel will end up > with a blank sheet. Using #ID is a practical workaround for this situation > but definitely not the most intuitive one. On the other hand, even most > NONMEM users are already used to some ‘tricks’, this issue still costs > unnecessary efforts and deepens the learning curve for NONMEM newbies. > Personally, it is surprising to see a well-maintained software lack support > for UTF-8 csv files, which are now well supported by all major csv viewers, > e.g. Editpad, Notepad++, open office, MS office. If this is a small fix > for NONMEM, I sincerely hope that it could become true. This feature might > potentially streamline NONMEM runs without adding # , as well as make > NONMEM learning more fun. > > > Thank you, > > > Mark > > On Mon, Jun 4, 2018 at 1:15 AM, Nick Holford <[email protected]> > wrote: > >> Hi, >> >> Excel can open files with ID as the first column header. You just have to >> agree to continue when Excel shows its warning messages. >> >> Personally I use #ID as the first column header which avoids the Excel >> warnings and also means I don’t need to specify an IGNORE character in >> NM-TRAN because # is the default. >> >> Best wishes, >> >> Nick >> >> >> >> -- >> >> Nick Holford, Professor Clinical Pharmacology >> >> Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A >> >> University of Auckland,85 Park Rd >> https://maps.google.com/?q=85+Park+Rd&entry=gmail&source=g,Private Bag >> 92019,Auckland,New Zealand >> >> office:+64(9)923-6730 mobile:NZ+64(21)46 23 53 FR+33(6)62 32 46 72 >> >> email: [email protected] >> >> http://holford.fmhs.auckland.ac.nz/ >> >> http://orcid.org/0000-0002-4031-2514 >> >> *Read the question, answer the question, attempt all questions* >> >> >> >> *From:* [email protected] <[email protected]> *On >> Behalf Of *HUI, Ka Ho >> *Sent:* Monday, June 4, 2018 4:48 AM >> *To:* Mark Tepeck <[email protected]>; [email protected] >> *Subject:* [FORGED] [NMusers] 回覆: [NMusers] [Suggest] NONMEM adds UTF-8 >> support for csv files >> >> >> >> Dear Mark >> >> >> >> Does it work if you don't put "ID" as the first column? Back to the days >> when I frequently used Excel, this had been the solution for me. >> >> >> >> Matthew >> >> >> >> ------ 原有訊息------ >> >> *寄件者： *Mark Tepeck >> >> *日期**: *2018年6月4日週一 10:41 >> >> *收件者： *[email protected]; >> >> *副本： * >> >> *主旨：*[NMusers] [Suggest] NONMEM adds UTF-8 support for csv files >> >> >> >> Hi NONMEM Users, >> >> >> >> As a PK scientist, I often manipulate csv files with Microsoft Excel, R, >> and NONMEM. As some of you may have seen, if the first record in a csv file >> is a string “ID”, Microsoft Excel would mistake it as an SYLK file which >> cannot be opened normally by Excel. One workaround is to use any strings >> other than ID, e.g. id, '‘ID’. However, this may cause new problems in some >> scenarios. Another potential solution is to use the R function >> readr::write_excel_csv ('dataframe.csv') to output a csv file with UTF-8 >> Byte order mark. Unfortunately, this csv file turned out to become >> unreadable for NONMEM although Excel likes it. Since waiting for bug fixing >> of Excel by Microsoft is not optimistic, I would truly appreciate it if >> NONMEM can add UTF-8 support. >> >> >> >> Thank you, >> >> >> >> Mark Tepeck >> > >

From: "Mark Tepeck" <[email protected]> To: "Nick Holford" <[email protected]>, [email protected] Sent: Monday, June 4, 2018 7:52:28 AM Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi All, Unfortunately, the latest release of MS Excel 2017 (build 9330.2087 for office 365) becomes more restrictive on SYLK files, which is apparently a step-back update, but really a typical Microsoft behavior ☹ . Even you agree to continue with the warning messages, the latest Excel will end up with a blank sheet. Using #ID is a practical workaround for this situation but definitely not the most intuitive one. On the other hand, even most NONMEM users are already used to some ‘tricks’, this issue still costs unnecessary efforts and deepens the learning curve for NONMEM newbies. Personally, it is surprising to see a well-maintained software lack support for UTF-8 csv files, which are now well supported by all major csv viewers, e.g. Editpad, Notepad++, open office, MS office. If this is a small fix for NONMEM, I sincerely hope that it could become true. This feature might potentially streamline NONMEM runs without adding # , as well as make NONMEM learning more fun. Thank you, Mark On Mon, Jun 4, 2018 at 1:15 AM, Nick Holford < [ mailto:[email protected] | [email protected] ] > wrote: Hi, Excel can open files with ID as the first column header. You just have to agree to continue when Excel shows its warning messages. Personally I use #ID as the first column header which avoids the Excel warnings and also means I don’t need to specify an IGNORE character in NM-TRAN because # is the default. Best wishes, Nick -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A University of Auckland, [ https://maps.google.com/?q=85+Park+Rd&entry=gmail&source=g | 85 Park Rd ] ,Private Bag 92019,Auckland,New Zealand office:+64(9)923-6730 mobile:NZ+64(21)46 23 53 FR+33(6)62 32 46 72 email: [ mailto:[email protected] | [email protected] ] [ http://holford.fmhs.auckland.ac.nz/ | http://holford.fmhs.auckland.ac.nz/ ] [ http://orcid.org/0000-0002-4031-2514 | http://orcid.org/0000-0002-4031-2514 ] Read the question, answer the question, attempt all questions From: [ mailto:[email protected] | [email protected] ] < [ mailto:[email protected] | [email protected] ] > On Behalf Of HUI, Ka Ho Sent: Monday, June 4, 2018 4:48 AM To: Mark Tepeck < [ mailto:[email protected] | [email protected] ] >; [ mailto:[email protected] | [email protected] ] Subject: [FORGED] [NMusers] 回覆 : [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Dear Mark Does it work if you don't put "ID" as the first column? Back to the days when I frequently used Excel, this had been the solution for me. Matthew ------ 原有訊息 ------ 寄件者： Mark Tepeck 日期 : 2018 年 6 月 4 日週一 10:41 收件者： [ mailto:[email protected] | [email protected] ] ; 副本： 主旨： [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi NONMEM Users, As a PK scientist, I often manipulate csv files with Microsoft Excel, R, and NONMEM. As some of you may have seen, if the first record in a csv file is a string “ID”, Microsoft Excel would mistake it as an SYLK file which cannot be opened normally by Excel. One workaround is to use any strings other than ID, e.g. id, '‘ID’. However, this may cause new problems in some scenarios. Another potential solution is to use the R function readr::write_excel_csv ('dataframe.csv') to output a csv file with UTF-8 Byte order mark. Unfortunately, this csv file turned out to become unreadable for NONMEM although Excel likes it. Since waiting for bug fixing of Excel by Microsoft is not optimistic, I would truly appreciate it if NONMEM can add UTF-8 support. Thank you, Mark Tepeck

________________________________________ From: owner-nmusers_at_globomaxnm.com [owner-nmusers_at_globomaxnm.com] on behalf of Sebastien Bihorel [sebastien.bihorel_at_cognigencorp.com] Sent: 04 June 2018 14:04 To: Mark Tepeck Cc: nmusers_at_globomaxnm.com Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi, Two things: - NONMEM does not require ID to be the first variable in $INPUT - the headers of your csv file do not have to match the variable names used in $INPUT So you have 2 easy workarounds for your issue: - as previously pointed out, you can use another first variable... a deletion flag variable or a study variable are as good candidates as ID for the 1st column of your data. - rename your ID variable as NMID in your csv file (or any alphanumeric combo that do not start with ID for that matter) Sebastien ________________________________ From: "Mark Tepeck" <mark.tepeck_at_gmail.com> To: "Nick Holford" <n.holford_at_auckland.ac.nz>, nmusers_at_globomaxnm.com Sent: Monday, June 4, 2018 7:52:28 AM Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi All, Unfortunately, the latest release of MS Excel 2017 (build 9330.2087 for office 365) becomes more restrictive on SYLK files, which is apparently a step-back update, but really a typical Microsoft behavior ☹. Even you agree to continue with the warning messages, the latest Excel will end up with a blank sheet. Using #ID is a practical workaround for this situation but definitely not the most intuitive one. On the other hand, even most NONMEM users are already used to some ‘tricks’, this issue still costs unnecessary efforts and deepens the learning curve for NONMEM newbies. Personally, it is surprising to see a well-maintained software lack support for UTF-8 csv files, which are now well supported by all major csv viewers, e.g. Editpad, Notepad++, open office, MS office. If this is a small fix for NONMEM, I sincerely hope that it could become true. This feature might potentially streamline NONMEM runs without adding # , as well as make NONMEM learning more fun. Thank you, Mark On Mon, Jun 4, 2018 at 1:15 AM, Nick Holford <n.holford_at_auckland.ac.nz<mailto:n.holford_at_auckland.ac.nz>> wrote: Hi, Excel can open files with ID as the first column header. You just have to agree to continue when Excel shows its warning messages. Personally I use #ID as the first column header which avoids the Excel warnings and also means I don’t need to specify an IGNORE character in NM-TRAN because # is the default. Best wishes, Nick -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A University of Auckland,85 Park https://maps.google.com/?q=85+Park+Rd&entry=gmail&source=g,Private Bag 92019,Auckland,New Zealand office:+64(9)923-6730 mobile:NZ+64(21)46 23 53 FR+33(6)62 32 46 72 email: n.holford_at_auckland.ac.nz<mailto:n.holford_at_auckland.ac.nz> http://holford.fmhs.auckland.ac.nz/ http://orcid.org/0000-0002-4031-2514 Read the question, answer the question, attempt all questions From: owner-nmusers_at_globomaxnm.com<mailto:owner-nmusers_at_globomaxnm.com> <owner-nmusers_at_globomaxnm.com<mailto:owner-nmusers_at_globomaxnm.com>> On Behalf Of HUI, Ka Ho Sent: Monday, June 4, 2018 4:48 AM To: Mark Tepeck <mark.tepeck_at_gmail.com<mailto:mark.tepeck_at_gmail.com>>; nmusers_at_globomaxnm.com<mailto:nmusers_at_globomaxnm.com> Subject: [FORGED] [NMusers] 回覆: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Dear Mark Does it work if you don't put "ID" as the first column? Back to the days when I frequently used Excel, this had been the solution for me. Matthew ------ 原有訊息------ 寄件者： Mark Tepeck 日期: 2018年6月4日週一 10:41 收件者： nmusers_at_globomaxnm.com<mailto:nmusers_at_globomaxnm.com>; 副本： 主旨：[NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi NONMEM Users, As a PK scientist, I often manipulate csv files with Microsoft Excel, R, and NONMEM. As some of you may have seen, if the first record in a csv file is a string “ID”, Microsoft Excel would mistake it as an SYLK file which cannot be opened normally by Excel. One workaround is to use any strings other than ID, e.g. id, '‘ID’. However, this may cause new problems in some scenarios. Another potential solution is to use the R function readr::write_excel_csv ('dataframe.csv') to output a csv file with UTF-8 Byte order mark. Unfortunately, this csv file turned out to become unreadable for NONMEM although Excel likes it. Since waiting for bug fixing of Excel by Microsoft is not optimistic, I would truly appreciate it if NONMEM can add UTF-8 support. Thank you, Mark Tepeck ******************************************************************************************************************** This message may contain confidential information. If you are not the intended recipient please inform the sender that you have received the message in error before deleting it. Please do not disclose, copy or distribute information in this e-mail or take any action in relation to its contents. To do so is strictly prohibited and may be unlawful. Thank you for your co-operation. NHSmail is the secure email and directory service available for all NHS staff in England and Scotland. NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and other accredited email services. For more information and to find out how you can switch, https://portal.nhs.net/help/joiningnhsmail

________________________________________ From: [email protected] [[email protected]] on behalf of Sebastien Bihorel [[email protected]] Sent: 04 June 2018 14:04 To: Mark Tepeck Cc: [email protected] Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi, Two things: - NONMEM does not require ID to be the first variable in $INPUT - the headers of your csv file do not have to match the variable names used in $INPUT So you have 2 easy workarounds for your issue: - as previously pointed out, you can use another first variable... a deletion flag variable or a study variable are as good candidates as ID for the 1st column of your data. - rename your ID variable as NMID in your csv file (or any alphanumeric combo that do not start with ID for that matter) Sebastien ________________________________ From: "Mark Tepeck" <[email protected]> To: "Nick Holford" <[email protected]>, [email protected] Sent: Monday, June 4, 2018 7:52:28 AM Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi All, Unfortunately, the latest release of MS Excel 2017 (build 9330.2087 for office 365) becomes more restrictive on SYLK files, which is apparently a step-back update, but really a typical Microsoft behavior ☹. Even you agree to continue with the warning messages, the latest Excel will end up with a blank sheet. Using #ID is a practical workaround for this situation but definitely not the most intuitive one. On the other hand, even most NONMEM users are already used to some ‘tricks’, this issue still costs unnecessary efforts and deepens the learning curve for NONMEM newbies. Personally, it is surprising to see a well-maintained software lack support for UTF-8 csv files, which are now well supported by all major csv viewers, e.g. Editpad, Notepad++, open office, MS office. If this is a small fix for NONMEM, I sincerely hope that it could become true. This feature might potentially streamline NONMEM runs without adding # , as well as make NONMEM learning more fun. Thank you, Mark On Mon, Jun 4, 2018 at 1:15 AM, Nick Holford <[email protected]<mailto:[email protected]>> wrote: Hi, Excel can open files with ID as the first column header. You just have to agree to continue when Excel shows its warning messages. Personally I use #ID as the first column header which avoids the Excel warnings and also means I don’t need to specify an IGNORE character in NM-TRAN because # is the default. Best wishes, Nick -- Nick Holford, Professor Clinical Pharmacology Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A University of Auckland,85 Park https://maps.google.com/?q=85+Park+Rd&entry=gmail&source=g,Private Bag 92019,Auckland,New Zealand office:+64(9)923-6730 mobile:NZ+64(21)46 23 53 FR+33(6)62 32 46 72 email: [email protected]<mailto:[email protected]> http://holford.fmhs.auckland.ac.nz/ http://orcid.org/0000-0002-4031-2514 Read the question, answer the question, attempt all questions From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> On Behalf Of HUI, Ka Ho Sent: Monday, June 4, 2018 4:48 AM To: Mark Tepeck <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Subject: [FORGED] [NMusers] 回覆: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Dear Mark Does it work if you don't put "ID" as the first column? Back to the days when I frequently used Excel, this had been the solution for me. Matthew ------ 原有訊息------ 寄件者： Mark Tepeck 日期: 2018年6月4日週一 10:41 收件者： [email protected]<mailto:[email protected]>; 副本： 主旨：[NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi NONMEM Users, As a PK scientist, I often manipulate csv files with Microsoft Excel, R, and NONMEM. As some of you may have seen, if the first record in a csv file is a string “ID”, Microsoft Excel would mistake it as an SYLK file which cannot be opened normally by Excel. One workaround is to use any strings other than ID, e.g. id, '‘ID’. However, this may cause new problems in some scenarios. Another potential solution is to use the R function readr::write_excel_csv ('dataframe.csv') to output a csv file with UTF-8 Byte order mark. Unfortunately, this csv file turned out to become unreadable for NONMEM although Excel likes it. Since waiting for bug fixing of Excel by Microsoft is not optimistic, I would truly appreciate it if NONMEM can add UTF-8 support. Thank you, Mark Tepeck ******************************************************************************************************************** This message may contain confidential information. If you are not the intended recipient please inform the sender that you have received the message in error before deleting it. Please do not disclose, copy or distribute information in this e-mail or take any action in relation to its contents. To do so is strictly prohibited and may be unlawful. Thank you for your co-operation. NHSmail is the secure email and directory service available for all NHS staff in England and Scotland. NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and other accredited email services. For more information and to find out how you can switch, https://portal.nhs.net/help/joiningnhsmail

On Tue, Jun 5, 2018 at 9:44 AM, STANDING, Joseph (GREAT ORMOND STREET HOSPITAL FOR CHILDREN NHS FOUNDATION TRUST) <[email protected]> wrote: > Dear Mark, > > Here is another workaround: abandon the use of Microsoft Excel (and all > possible Microsoft products) since this behaviour - new "improved" version > that you paid for - actually makes your life harder. I have also > encountered problems with Excel helpfully changing date format or time > zones without being asked - great of you are collating data recorded in > different time zones. If you must visualise your data in a spreadsheet > just use libre office, or some other open source solution, but in general > all data manipulations should be done in R not in a spreadsheet, so any > changes to the data are traceable. > > About 5 years ago I wiped Windows, installed Linux, and have never looked > back. RStudio, NONMEM, LaTex and all sensible software work just the same, > yet your computer now lasts much longer and runs faster because it is not > clogged up with inefficient operating system software. If you are > submitting to one of those backward journals that does not provide a LaTex > template, simply write your paper in Rmarkdown and output as a Word doc > (equations even come out OK these days). > > Much of the above goes for Mac operating system too unfortunately. > Upgrade the operating system and NONMEM stops working, students have had > terrible problems. If you must have a Mac because you like paying twice > the price for a metal case, wipe the operating system and put linux on. I > can highly recommend Lubuntu. > > BW, > > Joe > > > > > > ________________________________________ > From: [email protected] [[email protected]] on > behalf of Sebastien Bihorel [[email protected]] > Sent: 04 June 2018 14:04 > To: Mark Tepeck > Cc: [email protected] > Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files > > Hi, > > Two things: > - NONMEM does not require ID to be the first variable in $INPUT > - the headers of your csv file do not have to match the variable names > used in $INPUT > > So you have 2 easy workarounds for your issue: > - as previously pointed out, you can use another first variable... a > deletion flag variable or a study variable are as good candidates as ID for > the 1st column of your data. > - rename your ID variable as NMID in your csv file (or any alphanumeric > combo that do not start with ID for that matter) > > Sebastien > > ________________________________ > From: "Mark Tepeck" <[email protected]> > To: "Nick Holford" <[email protected]>, [email protected] > Sent: Monday, June 4, 2018 7:52:28 AM > Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files > > Hi All, > > Unfortunately, the latest release of MS Excel 2017 (build 9330.2087 for > office 365) becomes more restrictive on SYLK files, which is apparently a > step-back update, but really a typical Microsoft behavior ☹. Even you agree > to continue with the warning messages, the latest Excel will end up with a > blank sheet. Using #ID is a practical workaround for this situation but > definitely not the most intuitive one. On the other hand, even most NONMEM > users are already used to some ‘tricks’, this issue still costs unnecessary > efforts and deepens the learning curve for NONMEM newbies. Personally, it > is surprising to see a well-maintained software lack support for UTF-8 csv > files, which are now well supported by all major csv viewers, e.g. > Editpad, Notepad++, open office, MS office. If this is a small fix for > NONMEM, I sincerely hope that it could become true. This feature might > potentially streamline NONMEM runs without adding # , as well as make > NONMEM learning more fun. > > Thank you, > > Mark > > On Mon, Jun 4, 2018 at 1:15 AM, Nick Holford <[email protected] > <mailto:[email protected]>> wrote: > Hi, > Excel can open files with ID as the first column header. You just have to > agree to continue when Excel shows its warning messages. > Personally I use #ID as the first column header which avoids the Excel > warnings and also means I don’t need to specify an IGNORE character in > NM-TRAN because # is the default. > Best wishes, > Nick > > -- > Nick Holford, Professor Clinical Pharmacology > Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A > University of Auckland,85 Park https://maps.google.com/?q= 85+Park+Rd&entry=gmail&source=g>,Private Bag 92019,Auckland,New Zealand > office:+64(9)923-6730 mobile:NZ+64(21)46 23 53 FR+33(6)62 32 46 72 > email: [email protected]<mailto:[email protected]> > http://holford.fmhs.auckland.ac.nz/ > http://orcid.org/0000-0002-4031-2514 > Read the question, answer the question, attempt all questions > > From: [email protected]<mailto:[email protected]> < > [email protected]<mailto:[email protected]>> On > Behalf Of HUI, Ka Ho > Sent: Monday, June 4, 2018 4:48 AM > To: Mark Tepeck <[email protected]<mailto:[email protected]>>; > [email protected]<mailto:[email protected]> > Subject: [FORGED] [NMusers] 回覆: [NMusers] [Suggest] NONMEM adds UTF-8 > support for csv files > > Dear Mark > > Does it work if you don't put "ID" as the first column? Back to the days > when I frequently used Excel, this had been the solution for me. > > Matthew > > ------ 原有訊息------ > 寄件者： Mark Tepeck > 日期: 2018年6月4日週一 10:41 > 收件者： [email protected]<mailto:[email protected]>; > 副本： > 主旨：[NMusers] [Suggest] NONMEM adds UTF-8 support for csv files > > Hi NONMEM Users, > > As a PK scientist, I often manipulate csv files with Microsoft Excel, R, > and NONMEM. As some of you may have seen, if the first record in a csv file > is a string “ID”, Microsoft Excel would mistake it as an SYLK file which > cannot be opened normally by Excel. One workaround is to use any strings > other than ID, e.g. id, '‘ID’. However, this may cause new problems in some > scenarios. Another potential solution is to use the R function > readr::write_excel_csv ('dataframe.csv') to output a csv file with UTF-8 > Byte order mark. Unfortunately, this csv file turned out to become > unreadable for NONMEM although Excel likes it. Since waiting for bug fixing > of Excel by Microsoft is not optimistic, I would truly appreciate it if > NONMEM can add UTF-8 support. > > Thank you, > > Mark Tepeck > > > > ************************************************************ > ******************************************************** > > This message may contain confidential information. If you are not the > intended recipient please inform the > sender that you have received the message in error before deleting it. > Please do not disclose, copy or distribute information in this e-mail or > take any action in relation to its contents. To do so is strictly > prohibited and may be unlawful. Thank you for your co-operation. > > NHSmail is the secure email and directory service available for all NHS > staff in England and Scotland. NHSmail is approved for exchanging patient > data and other sensitive information with NHSmail and other accredited > email services. > > For more information and to find out how you can switch, > https://portal.nhs.net/help/joiningnhsmail > >

On Wed, Jun 6, 2018 at 12:04 AM Mark Tepeck <[email protected]> wrote: > Hi Joe and Sebastien, > > > Thank you for your feedback. Microsoft software is surely not my favorite > platform; However, I have to stick with MS due to the policy for the > organization computer I use. > > > I agree with Sebastien that there are some easy workarounds. It is indeed > easy on an individual basis, but it can be substantial efforts if > accumulated for all NONMEM users. NONMEM has more than a decade history; > why couldn’t it be made more robust, versatile and user-friendly? > > > Even if the new feature requested doesn’t benefit every end-user > immediately, it might be useful one day. UFT-8 BOM csv files are widely > supported by csv viewers, which speaks for their value in some scenarios. > Why couldn’t it be supported by NONMEM as well? The most successful > software are typically those constantly evolving and adapting to meet users > rather than those ignoring users feedback and calling them to make an > adaption. > > > Best, > > > Mark > > On Tue, Jun 5, 2018 at 9:44 AM, STANDING, Joseph (GREAT ORMOND STREET > HOSPITAL FOR CHILDREN NHS FOUNDATION TRUST) <[email protected]> > wrote: > >> Dear Mark, >> >> Here is another workaround: abandon the use of Microsoft Excel (and all >> possible Microsoft products) since this behaviour - new "improved" version >> that you paid for - actually makes your life harder. I have also >> encountered problems with Excel helpfully changing date format or time >> zones without being asked - great of you are collating data recorded in >> different time zones. If you must visualise your data in a spreadsheet >> just use libre office, or some other open source solution, but in general >> all data manipulations should be done in R not in a spreadsheet, so any >> changes to the data are traceable. >> >> About 5 years ago I wiped Windows, installed Linux, and have never looked >> back. RStudio, NONMEM, LaTex and all sensible software work just the same, >> yet your computer now lasts much longer and runs faster because it is not >> clogged up with inefficient operating system software. If you are >> submitting to one of those backward journals that does not provide a LaTex >> template, simply write your paper in Rmarkdown and output as a Word doc >> (equations even come out OK these days). >> >> Much of the above goes for Mac operating system too unfortunately. >> Upgrade the operating system and NONMEM stops working, students have had >> terrible problems. If you must have a Mac because you like paying twice >> the price for a metal case, wipe the operating system and put linux on. I >> can highly recommend Lubuntu. >> >> BW, >> >> Joe >> >> >> >> >> >> ________________________________________ >> From: [email protected] [[email protected]] on >> behalf of Sebastien Bihorel [[email protected]] >> Sent: 04 June 2018 14:04 >> To: Mark Tepeck >> Cc: [email protected] >> Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files >> >> Hi, >> >> Two things: >> - NONMEM does not require ID to be the first variable in $INPUT >> - the headers of your csv file do not have to match the variable names >> used in $INPUT >> >> So you have 2 easy workarounds for your issue: >> - as previously pointed out, you can use another first variable... a >> deletion flag variable or a study variable are as good candidates as ID for >> the 1st column of your data. >> - rename your ID variable as NMID in your csv file (or any alphanumeric >> combo that do not start with ID for that matter) >> >> Sebastien >> >> ________________________________ >> From: "Mark Tepeck" <[email protected]> >> To: "Nick Holford" <[email protected]>, [email protected] >> Sent: Monday, June 4, 2018 7:52:28 AM >> Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files >> >> Hi All, >> >> Unfortunately, the latest release of MS Excel 2017 (build 9330.2087 for >> office 365) becomes more restrictive on SYLK files, which is apparently a >> step-back update, but really a typical Microsoft behavior ☹. Even you agree >> to continue with the warning messages, the latest Excel will end up with a >> blank sheet. Using #ID is a practical workaround for this situation but >> definitely not the most intuitive one. On the other hand, even most NONMEM >> users are already used to some ‘tricks’, this issue still costs unnecessary >> efforts and deepens the learning curve for NONMEM newbies. Personally, it >> is surprising to see a well-maintained software lack support for UTF-8 csv >> files, which are now well supported by all major csv viewers, e.g. >> Editpad, Notepad++, open office, MS office. If this is a small fix for >> NONMEM, I sincerely hope that it could become true. This feature might >> potentially streamline NONMEM runs without adding # , as well as make >> NONMEM learning more fun. >> >> Thank you, >> >> Mark >> >> On Mon, Jun 4, 2018 at 1:15 AM, Nick Holford <[email protected] >> <mailto:[email protected]>> wrote: >> Hi, >> Excel can open files with ID as the first column header. You just have to >> agree to continue when Excel shows its warning messages. >> Personally I use #ID as the first column header which avoids the Excel >> warnings and also means I don’t need to specify an IGNORE character in >> NM-TRAN because # is the default. >> Best wishes, >> Nick >> >> -- >> Nick Holford, Professor Clinical Pharmacology >> Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A >> University of Auckland,85 Park Rd >> https://maps.google.com/?q=Auckland,85+Park+Rd&entry=gmail&source=g< >> https://maps.google.com/?q=85+Park+Rd&entry=gmail&source=g>,Private Bag >> 92019,Auckland,New Zealand >> office:+64(9)923-6730 <+64%209-923%206730> mobile:NZ+64(21)46 23 53 >> FR+33(6)62 >> 32 46 72 <+33%206%2062%2032%2046%2072> >> email: [email protected]<mailto:[email protected]> >> http://holford.fmhs.auckland.ac.nz/ >> http://orcid.org/0000-0002-4031-2514 >> Read the question, answer the question, attempt all questions >> >> From: [email protected]<mailto:[email protected]> < >> [email protected]<mailto:[email protected]>> On >> Behalf Of HUI, Ka Ho >> Sent: Monday, June 4, 2018 4:48 AM >> To: Mark Tepeck <[email protected]<mailto:[email protected]>>; >> [email protected]<mailto:[email protected]> >> Subject: [FORGED] [NMusers] 回覆: [NMusers] [Suggest] NONMEM adds UTF-8 >> support for csv files >> >> Dear Mark >> >> Does it work if you don't put "ID" as the first column? Back to the days >> when I frequently used Excel, this had been the solution for me. >> >> Matthew >> >> ------ 原有訊息------ >> 寄件者： Mark Tepeck >> 日期: 2018年6月4日週一 10:41 >> 收件者： [email protected]<mailto:[email protected]>; >> 副本： >> 主旨：[NMusers] [Suggest] NONMEM adds UTF-8 support for csv files >> >> Hi NONMEM Users, >> >> As a PK scientist, I often manipulate csv files with Microsoft Excel, R, >> and NONMEM. As some of you may have seen, if the first record in a csv file >> is a string “ID”, Microsoft Excel would mistake it as an SYLK file which >> cannot be opened normally by Excel. One workaround is to use any strings >> other than ID, e.g. id, '‘ID’. However, this may cause new problems in some >> scenarios. Another potential solution is to use the R function >> readr::write_excel_csv ('dataframe.csv') to output a csv file with UTF-8 >> Byte order mark. Unfortunately, this csv file turned out to become >> unreadable for NONMEM although Excel likes it. Since waiting for bug fixing >> of Excel by Microsoft is not optimistic, I would truly appreciate it if >> NONMEM can add UTF-8 support. >> >> Thank you, >> >> Mark Tepeck >> >> >> >> >> ******************************************************************************************************************** >> >> This message may contain confidential information. If you are not the >> intended recipient please inform the >> sender that you have received the message in error before deleting it. >> Please do not disclose, copy or distribute information in this e-mail or >> take any action in relation to its contents. To do so is strictly >> prohibited and may be unlawful. Thank you for your co-operation. >> >> NHSmail is the secure email and directory service available for all NHS >> staff in England and Scotland. NHSmail is approved for exchanging patient >> data and other sensitive information with NHSmail and other accredited >> email services. >> >> For more information and to find out how you can switch, >> https://portal.nhs.net/help/joiningnhsmail >> >> >

From: [email protected] [mailto:[email protected]] On Behalf Of Mark Tepeck Sent: 05 June 2018 22:56 To: [email protected] Cc: Sebastien Bihorel <[email protected]>; STANDING, Joseph (GREAT ORMOND STREET HOSPITAL FOR CHILDREN NHS FOUNDATION TRUST) <[email protected]> Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi Joe and Sebastien, Thank you for your feedback. Microsoft software is surely not my favorite platform; However, I have to stick with MS due to the policy for the organization computer I use. I agree with Sebastien that there are some easy workarounds. It is indeed easy on an individual basis, but it can be substantial efforts if accumulated for all NONMEM users. NONMEM has more than a decade history; why couldn’t it be made more robust, versatile and user-friendly? Even if the new feature requested doesn’t benefit every end-user immediately, it might be useful one day. UFT-8 BOM csv files are widely supported by csv viewers, which speaks for their value in some scenarios. Why couldn’t it be supported by NONMEM as well? The most successful software are typically those constantly evolving and adapting to meet users rather than those ignoring users feedback and calling them to make an adaption. Best, Mark

From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Mark Tepeck Sent: Tuesday, June 05, 2018 2:56 PM To: [email protected]<mailto:[email protected]> Cc: Sebastien Bihorel; STANDING, Joseph (GREAT ORMOND STREET HOSPITAL FOR CHILDREN NHS FOUNDATION TRUST) Subject: Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files Hi Joe and Sebastien, Thank you for your feedback. Microsoft software is surely not my favorite platform; However, I have to stick with MS due to the policy for the organization computer I use. I agree with Sebastien that there are some easy workarounds. It is indeed easy on an individual basis, but it can be substantial efforts if accumulated for all NONMEM users. NONMEM has more than a decade history; why couldn’t it be made more robust, versatile and user-friendly? Even if the new feature requested doesn’t benefit every end-user immediately, it might be useful one day. UFT-8 BOM csv files are widely supported by csv viewers, which speaks for their value in some scenarios. Why couldn’t it be supported by NONMEM as well? The most successful software are typically those constantly evolving and adapting to meet users rather than those ignoring users feedback and calling them to make an adaption. Best, Mark

On Wed, Jun 6, 2018 at 1:45 PM, Bauer, Robert <[email protected]> wrote: > Mark: > > NONMEM does not use or interpret extended UTF-8 code, but perhaps you mean > that NONMEM could filter out bytes>127 when reading in data files that are > UTF-8 encoded, and process only bytes <=127. This can certainly be done, > and I can add it to the list of improvements for the next release. > > > > Robert J. Bauer, Ph.D. > > Senior Director > > Pharmacometrics R&D > > ICON Early Phase > > 820 W. Diamond Avenue > https://maps.google.com/?q=820+W.+Diamond+Avenue+%0D%0A+Suite+100+%0D%0A+Gaithersburg,+MD+20878+%0D%0A+Office:+(215&entry=gmail&source=g > > Suite 100 > https://maps.google.com/?q=820+W.+Diamond+Avenue+%0D%0A+Suite+100+%0D%0A+Gaithersburg,+MD+20878+%0D%0A+Office:+(215&entry=gmail&source=g > > Gaithersburg, MD 20878 > https://maps.google.com/?q=820+W.+Diamond+Avenue+%0D%0A+Suite+100+%0D%0A+Gaithersburg,+MD+20878+%0D%0A+Office:+(215&entry=gmail&source=g > > Office: (215 > https://maps.google.com/?q=820+W.+Diamond+Avenue+%0D%0A+Suite+100+%0D%0A+Gaithersburg,+MD+20878+%0D%0A+Office:+(215&entry=gmail&source=g) > 616-6428 > > Mobile: (925) 286-0769 > > [email protected] > > www.iconplc.com > > > > *From:* [email protected] [mailto:[email protected] > <[email protected]>] *On Behalf Of *Mark Tepeck > *Sent:* Tuesday, June 05, 2018 2:56 PM > *To:* [email protected] > *Cc:* Sebastien Bihorel; STANDING, Joseph (GREAT ORMOND STREET HOSPITAL > FOR CHILDREN NHS FOUNDATION TRUST) > *Subject:* Re: [NMusers] [Suggest] NONMEM adds UTF-8 support for csv files > > > > Hi Joe and Sebastien, > > > > Thank you for your feedback. Microsoft software is surely not my favorite > platform; However, I have to stick with MS due to the policy for the > organization computer I use. > > > > I agree with Sebastien that there are some easy workarounds. It is indeed > easy on an individual basis, but it can be substantial efforts if > accumulated for all NONMEM users. NONMEM has more than a decade history; > why couldn’t it be made more robust, versatile and user-friendly? > > > > Even if the new feature requested doesn’t benefit every end-user > immediately, it might be useful one day. UFT-8 BOM csv files are widely > supported by csv viewers, which speaks for their value in some scenarios. > Why couldn’t it be supported by NONMEM as well? The most successful > software are typically those constantly evolving and adapting to meet users > rather than those ignoring users feedback and calling them to make an > adaption. > > > > Best, > > > > Mark > > > > >